Skip to main content

​
Overview

The Moda Python SDK provides automatic instrumentation for your LLM applications with built-in conversation threading. Every LLM call is automatically tracked with a stable moda.conversation_id that groups multi-turn conversations together.

​
Installation

pip install moda-ai
This installs the core SDK along with instrumentation for OpenAI and Anthropic.

​
Quick Start

import moda
from openai import OpenAI

moda.init("YOUR_MODA_API_KEY")

# Set conversation ID for your session (recommended)
moda.conversation_id = "session_" + session_id

client = OpenAI()
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}]
)

moda.flush()
Initialization is synchronous in Python. Call moda.init(...) once at startup before making LLM calls to ensure instrumentation is active.

​
Conversation Tracking

For production use, explicitly set a conversation ID to group related LLM calls: 
import moda

moda.init("YOUR_MODA_API_KEY")

# Property-style (recommended)
moda.conversation_id = "support_ticket_123"

client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "I need help"}]
)

client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "user", "content": "I need help"},
        {"role": "assistant", "content": "I'd be happy to help..."},
        {"role": "user", "content": "Order #12345"}
    ]
)

# Both calls share the same conversation_id
moda.conversation_id = None  # clear when done

​
Setting User ID

Associate LLM calls with specific users for per-user analytics: 
moda.user_id = "user_12345"

client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello"}]
)

moda.user_id = None  # clear when done

​
Scoped Context Managers

For scoped context (useful in request handlers): 
from moda import set_conversation_id, set_user_id

# Group specific calls under a custom conversation ID
with set_conversation_id("support-ticket-123"):
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": "I need help"}]
    )

# Attach user attribution
with set_user_id("user-456"):
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": "Hello"}]
    )

​
Direct Setters

For cases where context managers aren’t suitable (e.g., setting once at request start): 
from moda import set_conversation_id_value, set_user_id_value

# Set values that persist until cleared
set_conversation_id_value("session-123")
set_user_id_value("user-456")

# Make multiple calls...
response1 = client.chat.completions.create(...)
response2 = client.chat.completions.create(...)

# Clear when done
set_conversation_id_value(None)
set_user_id_value(None)

​
Reading Current Context

from moda import get_conversation_id, get_user_id

# Get current values
current_conv = get_conversation_id()
current_user = get_user_id()
print(f"Conversation: {current_conv}, User: {current_user}")

​
Computing Conversation ID

You can manually compute a conversation ID from messages: 
from moda import compute_conversation_id

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Hello!"}
]

# Returns conv_[16-char-hash] based on system prompt + first user message
conv_id = compute_conversation_id(messages)

​
Automatic Fallback

If you don’t set a conversation ID, the SDK automatically computes a stable one based on:
  • The first user message in the conversation
  • The system prompt (if present)
This works well for prototyping and simple use cases: 
import moda
from openai import OpenAI

moda.init("YOUR_MODA_API_KEY")
client = OpenAI()

# Turn 1
messages = [{"role": "user", "content": "What is Python?"}]
response = client.chat.completions.create(model="gpt-4o", messages=messages)

# Turn 2 - automatically has the same conversation_id
messages.append({"role": "assistant", "content": response.choices[0].message.content})
messages.append({"role": "user", "content": "How do I install it?"})
response = client.chat.completions.create(model="gpt-4o", messages=messages)

# Both calls share the same conversation_id in Moda
For production applications, explicit conversation IDs are recommended as they provide:
  • Predictable grouping regardless of message content
  • Integration with your existing session/thread identifiers
  • Easier debugging and correlation with your application logs

​
Configuration

​
Environment Variables

VariableDescription
MODA_API_KEYYour Moda API key
MODA_BASE_URLCustom ingest endpoint (optional)
MODA_HEADERSCustom headers sent with telemetry requests (optional)

​
Programmatic Configuration

import moda

moda.init(
    api_key="YOUR_MODA_API_KEY",
    app_name="my-chatbot",           # Optional: name your application
    endpoint="https://custom.endpoint/v1/traces"  # Optional: custom endpoint
)

​
Advanced Configuration

The SDK supports additional configuration options: 
import moda

moda.init(
    api_key="YOUR_MODA_API_KEY",
    app_name="my-app",

    # Enable/disable instrumentation
    enabled=True,

    # Send spans immediately instead of batching
    disable_batch=False,

    # Additional resource attributes
    resource_attributes={
        "deployment.environment": "production",
    },

    # Filter which providers are instrumented (see below)
    instruments=None,                 # Set of instruments to enable (None = all)
    block_instruments=None,           # Set of instruments to disable

    # Additional headers sent with telemetry requests
    headers={},
)

​
Filtering Instruments

Control which LLM providers are instrumented: 
from moda import Instruments
import moda

# Only instrument OpenAI
moda.init(
    api_key="YOUR_KEY",
    instruments={Instruments.OPENAI}
)

# Instrument everything except Anthropic
moda.init(
    api_key="YOUR_KEY",
    block_instruments={Instruments.ANTHROPIC}
)
The Instruments enum supports a wide range of providers and frameworks:
CategoryInstruments
LLM ProvidersOPENAI, ANTHROPIC, COHERE, MISTRAL, GROQ, OLLAMA, BEDROCK, VERTEXAI, TOGETHER, REPLICATE, SAGEMAKER, WATSONX, GOOGLE_GENERATIVEAI, WRITER, ALEPHALPHA
FrameworksLANGCHAIN, LLAMA_INDEX, HAYSTACK, CREWAI, OPENAI_AGENTS, MCP, AGNO
Vector DatabasesCHROMA, MILVUS, PINECONE, QDRANT, WEAVIATE, LANCEDB, MARQO

​
Supported Providers

The SDK automatically instruments:
ProviderPackage
OpenAImoda-openai (included)
Anthropicmoda-anthropic (included)

​
OpenRouter Support

OpenRouter provides access to multiple LLM providers through a unified API. Since OpenRouter uses an OpenAI-compatible interface, it works automatically with the Moda SDK: 
import moda
from openai import OpenAI

moda.init("YOUR_MODA_API_KEY")

# Configure OpenAI client to use OpenRouter
openrouter = OpenAI(
    base_url="https://openrouter.ai/api/v1",
    api_key="YOUR_OPENROUTER_API_KEY",
    default_headers={
        "HTTP-Referer": "https://your-app.com",  # Optional: for rankings
        "X-Title": "Your App Name",               # Optional: for rankings
    },
)

moda.conversation_id = "openrouter_session_123"

# Use any model available on OpenRouter
response = openrouter.chat.completions.create(
    model="anthropic/claude-3.5-sonnet",  # Or any OpenRouter model
    messages=[{"role": "user", "content": "Hello!"}]
)

# Also works with OpenAI models via OpenRouter
gpt_response = openrouter.chat.completions.create(
    model="openai/gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}]
)
OpenRouter model names use the format provider/model-name. See the OpenRouter models page for all available models.

​
Verifying Data in Moda

After setting up, verify that data is flowing correctly:
  1. Make a few LLM calls
  2. Call moda.flush() to ensure data is sent
  3. Check the Moda dashboard — conversations should appear within seconds
  4. Verify that multi-turn conversations share the same conversation ID

​
Data Captured

The SDK captures:
AttributeDescription
moda.conversation_idStable ID grouping multi-turn conversations
moda.user_idUser identifier (when set)
llm.vendorLLM provider (e.g., “openai”, “anthropic”)
llm.request.typeRequest type (e.g., “chat”, “completion”)
llm.request.modelRequested model name
llm.response.modelActual model used in response
llm.promptsUser and system messages
llm.completionsAssistant responses
llm.usage.prompt_tokensInput token count
llm.usage.completion_tokensOutput token count
llm.usage.total_tokensTotal token count
llm.usage.reasoning_tokensReasoning token count (extended thinking)

​
API Reference

​
Core Functions

FunctionDescription
moda.init(api_key, **options)Initialize the SDK
moda.flush()Force flush pending telemetry

​
Context Properties

PropertyDescription
moda.conversation_idGet/set global conversation ID
moda.user_idGet/set global user ID

​
Context Managers

FunctionDescription
set_conversation_id(id)Context manager for scoped conversation ID
set_user_id(id)Context manager for scoped user ID

​
Direct Functions

FunctionDescription
set_conversation_id_value(id)Set conversation ID without context manager
set_user_id_value(id)Set user ID without context manager
get_conversation_id()Get current conversation ID
get_user_id()Get current user ID
compute_conversation_id(messages)Compute ID from message history

​
Troubleshooting

Conversation IDs not grouping correctly?
  • Use explicit moda.conversation_id instead of relying on auto-compute
  • If using auto-compute, ensure the first user message stays the same across turns
  • Check if system prompts are changing between calls
Data not appearing in Moda?
  • Call moda.flush() before your program exits
  • Check that your API key is correct
  • Verify network connectivity to the ingest endpoint
Import errors?
  • Make sure you installed moda-ai
  • Check that your Python version is 3.10 or higher