Agentic Observability Platforms [2026 Cheat Sheet]

As of May 08, 2026, the practical gap between agentic observability platforms is no longer “who has tracing?” but “which operating model fits your team?” LangSmith, Arize Phoenix, and W&B Weave all cover traces and evals; the differences show up in instrumentation style, workflow shape, and how naturally each product fits your existing stack. This cheat sheet focuses on those decision points, plus copy-ready setup references you can filter live.

• All three platforms now cover tracing, evaluation, and prompt iteration loops.
• Phoenix is the most explicit bet on OpenTelemetry and OpenInference.
• LangSmith is the most opinionated end-to-end agent application workflow.
• Weave feels strongest when your team already operates inside W&B.
• Before shipping traces, scrub sensitive fields with a Data Masking Tool.

Decision matrix below is an inference from official docs, not vendor marketing copy.

When to Choose Each

Choose LangSmith when:

• You want tracing, datasets, evaluators, prompts, and experiments to live in one workflow.
• You are already building with LangChain or LangGraph and want fast instrumentation via environment variables and wrappers.
• You care about both offline evaluation and online evaluation in a single product model.
• You want a more app-centric workflow than a raw telemetry-centric one.

Choose Phoenix when:

• You want the most explicit OTLP, OpenTelemetry, and OpenInference path.
• You want open-source posture and documented self-hosting on Docker, Kubernetes, or your own cloud.
• You want prompt replay, datasets, and experiments without giving up an infra-friendly tracing model.
• Your team prefers instrumentation that is easy to move or standardize across providers.

Choose Weave when:

• Your org already uses Weights & Biases teams, entities, or evaluation workflows.
• You want function-level tracking with weave.op and project-centric logging via weave.init().
• You want prompts, versions, traces, scorers, and feedback to sit next to the rest of your W&B work.
• You need a lighter bridge from LLM app tracing into an existing W&B culture.

Searchable Reference

Use the filter box to narrow commands, snippets, or platform notes. Keyboard shortcuts are wired below.

Install and authenticate

pip install -U langsmith openai

export LANGSMITH_TRACING=true
export LANGSMITH_API_KEY="<your-langsmith-api-key>"
export OPENAI_API_KEY="<your-openai-api-key>"
export LANGSMITH_PROJECT="my-app"

pip install arize-phoenix openinference-instrumentation-openai openai

pip install weave openai

export WANDB_API_KEY="<your_api_key>"

Basic tracing

from openai import OpenAI
from langsmith.wrappers import wrap_openai
from langsmith import traceable

client = wrap_openai(OpenAI())

@traceable
def assistant(question: str) -> str:
    response = client.chat.completions.create(
        model="gpt-4.1-mini",
        messages=[{"role": "user", "content": question}],
    )
    return response.choices[0].message.content

from phoenix.otel import register

tracer_provider = register(
    project_name="my-llm-app",
    auto_instrument=True,
)

import weave
from openai import OpenAI

client = OpenAI()
weave.init('your-team/traces-quickstart')

@weave.op()
def ask_model(prompt: str):
    return client.chat.completions.create(
        model='gpt-4o',
        messages=[{'role': 'user', 'content': prompt}],
    )

CLI and query workflows

langsmith project list
langsmith trace list --project my-app --limit 5
langsmith run list --project my-app --run-type llm --include-metadata
langsmith dataset list
langsmith experiment list --dataset my-eval-set

npm install @arizeai/openinference-instrumentation-openai

npm install weave openai

Configuration

LangSmith configuration notes

• LANGSMITH_TRACING enables tracing.
• LANGSMITH_PROJECT routes traces into a named project.
• LANGSMITH_WORKSPACE_ID is relevant when an API key is linked to multiple workspaces.
• LANGSMITH_ENDPOINT matters for self-hosted or hybrid deployments.

Phoenix configuration notes

• Phoenix docs now strongly emphasize using phoenix.otel and OTEL-aware defaults.
• auto_instrument=True activates installed OpenInference instrumentors automatically.
• For cloud setups, docs show PHOENIX_API_KEY and PHOENIX_COLLECTOR_ENDPOINT as the core connection values.
• If you batch spans, flush on shutdown so data is not left in the exporter queue.

Weave configuration notes

• WANDB_API_KEY can log you in non-interactively.
• weave.init('entity/project') is the project-routing primitive that matters first.
• WEAVE_PARALLELISM controls worker parallelism.
• WEAVE_PRINT_CALL_LINK=false disables terminal call-link output.

# Weave runtime environment variables
export WEAVE_PARALLELISM=10
export WEAVE_PRINT_CALL_LINK=false

Advanced Usage

Evaluation strategy that travels across all three

• Start with a small hand-built dataset of failures, not a giant synthetic benchmark.
• Separate offline evaluation from online monitoring so regressions and live drift are not mixed together.
• Track retrieval quality, tool selection, formatting validity, and final answer quality as separate signals.
• Promote bad production traces into eval datasets quickly; all three platforms support some version of that feedback loop.

Where each platform pulls ahead in advanced workflows

• LangSmith: strongest when you want traces, evaluators, prompts, and experiment comparison in one application workflow.
• Phoenix: strongest when you want open instrumentation and portability around OTLP and OpenInference.
• Weave: strongest when you want LLM observability to feel like an extension of existing W&B evaluation and project habits.

Security and retention checklist

• Decide whether prompts, retrieved chunks, and tool arguments are safe to store before you enable tracing broadly.
• Mask secrets and identifiers at the edge, not after ingestion.
• Keep one documented policy for trace retention, evaluator retention, and dataset retention.
• If you share snippets across teams, run them through the Code Formatter before pasting into internal docs or runbooks.

Official Sources

• LangSmith docs overview
• LangSmith tracing quickstart
• LangSmith evaluation docs
• LangSmith CLI reference
• Arize Phoenix docs overview
• Phoenix OTEL setup
• Arize platform overview including Phoenix and AX
• W&B Weave overview
• W&B Weave quickstart
• Weave environment variables