Endpoint details
- Path:
/otel/v1/traces - Method:
POST - Content-Type:
application/x-protobuf - Base URL: The base URL for the OTel trace endpoint depends on your W&B deployment type:
- Multi-tenant Cloud:
https://trace.wandb.ai/otel/v1/traces. - Dedicated Cloud and Self-Managed instances:
https://<your-subdomain>.wandb.io/traces/otel/v1/traces.
<your-subdomain> with your organization’s unique W&B domain, for example, acme.wandb.io.
Authentication and routing
Weave uses thewandb-api-key header to authenticate requests and resource attributes on your TracerProvider to route spans to the correct entity and project. Pass your W&B API key in the wandb-api-key header, then specify the following keys as OpenTelemetry Resource attributes in your TracerProvider class:
wandb.entity: Your W&B team or user name.wandb.project: The project name to send traces to.
Examples
The following examples show how to send OpenTelemetry traces to Weave using Python and TypeScript. Each example covers a different approach: using the OpenInference instrumentation library, using OpenLLMetry instrumentation, or using the OpenTelemetry SDK directly without an instrumentation package. Before running the following code samples, set the following fields:WANDB_API_KEY: You can get this from User Settings.- Entity: You can only log traces to the project under a team/entity that you have access to. To find your entity name, visit your W&B dashboard and check the Teams field in the left sidebar.
- Project name: Choose a name.
OPENAI_API_KEY: You can obtain this from the OpenAI dashboard.
OpenInference instrumentation
OpenInference is an open source instrumentation library from Arize AI that captures LLM calls as OpenTelemetry spans. This example shows how to use the OpenAI instrumentation. Additional instrumentations are available in the official repository. First, install the required dependencies:- Python
- TypeScript
Paste the following code into a Python file such as Run the code:
openinference_example.py:OpenLLMetry instrumentation
OpenLLMetry is an open source observability library from Traceloop that provides OpenTelemetry instrumentation for popular LLM providers and frameworks. The following example shows how to use its OpenAI instrumentation. Additional examples are available in the OpenLLMetry repository. First, install the required dependencies:- Python
- TypeScript
Paste the following code into a Python file such as Run the code:
openllmetry_example.py. This is the same code as the preceding example, except the OpenAIInstrumentor is imported from opentelemetry.instrumentation.openai instead of openinference.instrumentation.openai:Without instrumentation
If you prefer to use OTel directly instead of an instrumentation package, you can do so. This approach gives you full control over which attributes are set on each span. Weave parses span attributes according to the OpenTelemetry semantic conventions described at https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-spans/. First, install the required dependencies:- Python
- TypeScript
Paste the following code into a Python file such as Run the code:
opentelemetry_example.py:gen_ai and openinference to determine which convention to apply, if any, when interpreting the trace. If neither key is detected, then all span attributes are visible in the trace view. The full span is available in the side panel when you select a trace.
Use an OpenTelemetry Collector
The previous examples export traces directly from your application to Weave. In production, you can use an OpenTelemetry Collector as an intermediary between your application and Weave. The collector receives traces from your app, then forwards them to one or more backends. This pattern centralizes authentication, batching, and routing logic outside of your application code, and lets you fan traces out to multiple observability backends from a single pipeline.Set up a collector
This section walks through running a local OpenTelemetry Collector in Docker and configuring an application to send traces to it. The following example shows how to:- Set up a Docker configuration file that deploys a local server (collector) that listens for OTLP traces, batches them, and forwards them to Weave.
- Locally run the collector using Docker.
- Send a basic call to OpenAI that forwards traces to the collector running in the Docker container.
collector-config.yaml file that configures the collector to receive OTLP traces and export them to Weave:
collector-config.yaml
- Listens for OTLP traces on port
4318(HTTP). - Exports traces to Weave’s OTLP endpoint using the
wandb-api-keyheader, reading the endpoint URL fromWANDB_OTLP_ENDPOINTand the API key fromWANDB_API_KEY. - Sets
wandb.entityandwandb.projectas resource attributes using theresourceprocessor, reading values fromDEFAULT_WANDB_ENTITYandDEFAULT_WANDB_PROJECT. Theinsertaction injects these attributes only if your application code does not already set them. - Enables the exporter’s built-in
sending_queuewith batching to reduce network overhead.
OTEL_EXPORTER_OTLP_ENDPOINT environment variable. The OTel SDK reads this variable automatically, so you don’t need to pass the endpoint to the exporter.
If you set wandb.entity or wandb.project as resource attributes in your application’s TracerProvider, they take precedence over the defaults defined in the collector config.
OpenAIInstrumentor wraps OpenAI calls, creates traces, and exports them to the collector. The collector handles authentication and routing to Weave.
After running the script, you can view the traces in the Weave UI.
To send traces to additional backends, add more exporters and include them in the service.pipelines.traces.exporters list. For example, you can export to both Weave and Jaeger from the same Collector instance.
Organize OTel traces into threads
Weave threads let you group related traces so you can analyze multi-turn conversations or user sessions as a single unit. Add specific span attributes to organize your OpenTelemetry traces into threads, then use Weave’s Thread UI to analyze related operations such as multi-turn conversations or user sessions. Add the following attributes to your OTel spans to enable thread grouping:wandb.thread_id: Groups spans into a specific thread.wandb.is_turn: Marks a span as a conversation turn (appears as a row in the thread view).
wandb.thread_id to group related operations and wandb.is_turn to mark high-level operations that appear as rows in the thread view.
Initial set up
Initial set up
Use this configuration to run these examples:
Trace a basic single-turn thread
Trace a basic single-turn thread
Trace a multi-turn conversation sharing one thread ID
Trace a multi-turn conversation sharing one thread ID
Trace deeply nested operations and mark only the outermost span as a turn
Trace deeply nested operations and mark only the outermost span as a turn
Trace background operations that belong to a thread but aren't turns
Trace background operations that belong to a thread but aren't turns
thread_id and each turn appears as a separate row.
Attribute mappings
Weave maps OpenTelemetry span attributes from various instrumentation frameworks to its internal data model. This mapping means you don’t need to rename or transform attributes from your existing instrumentation to get a rich view in Weave. When multiple attribute names map to the same field, Weave applies them in priority order, which lets frameworks coexist in the same traces.Supported frameworks
Weave supports attribute conventions from the following observability frameworks and SDKs:- OpenTelemetry GenAI: Standard semantic conventions for generative AI (
gen_ai.*). - OpenInference: Arize AI’s instrumentation library (
input.value,output.value,llm.*,openinference.*). - Vercel AI SDK: Vercel’s AI SDK attributes (
ai.prompt,ai.response,ai.model.*,ai.usage.*). - MLflow: MLflow tracking attributes (
mlflow.spanInputs,mlflow.spanOutputs). - Traceloop: OpenLLMetry instrumentation (
traceloop.entity.*,traceloop.span.kind). - Google Vertex AI: Vertex AI agent attributes (
gcp.vertex.agent.*). - OpenLit: OpenLit observability attributes (
gen_ai.content.completion). - Langfuse: Langfuse tracing attributes (
langfuse.startTime,langfuse.endTime).