Arize Phoenix

Heritage explains a lot of the design. Arize built ML observability for tabular and embedding models before LLM applications were the market, and Phoenix inherits that lineage: embedding drift views, cluster analysis and retrieval quality inspection that most LLM-native tools never got around to building. The tracing layer is OpenTelemetry throughout, using the OpenInference semantic conventions (Apache 2.0) to describe what a span means when it happens to be an LLM call, a retriever hit or a tool invocation.

In practice it starts very small. pip install arize-phoenix, call px.launch_app() in a notebook, point instrumentation at it and spans appear within a minute with no account and no infrastructure. Move to Docker or Kubernetes when that stops being enough. Evaluation ships graders for hallucination, relevance, toxicity and QA correctness, and Arize publishes benchmark numbers for the graders themselves, a level of self-scrutiny worth rewarding in a field full of unvalidated LLM judges. Datasets and experiments handle offline comparison, the playground replays a captured trace against a different model or prompt, and a remote MCP server exposes the trace store to coding agents.

The licence is the thing to settle before adopting. Phoenix is Elastic License 2.0: source-available, free to self-host with no usage caps and no feature gates, but not OSI-approved, and you may not offer it as a hosted service to third parties. Some legal teams treat that as disqualifying regardless of practical impact. The commercial path is Arize AX, free at 25k spans a month with 15-day retention, $50 a month at Pro for 50k spans and 10 GB, quoted at Enterprise. If the requirement is a fully OSI-licensed production trace store, Langfuse under MIT is the closer fit. If the requirement is OTel correctness and evaluator rigour during development, this is the better tool.