Observability & Evaluation
26 agents tracked · ranked by trust score · Updated 2026-10-13 12:10 UTC · the three strongest right now:
Grade mix26 agents
5 A9 B6 C6 D
| # | Agent | Trust | Stars | Language |
|---|---|---|---|---|
| 1 | Promptfoo A Listed Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic. | 92.1 | 26.0k | TypeScript |
| 2 | MLflow A Listed The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data. | 89.6 | 28.4k | Python |
| 3 | LangWatch A Listed Open-Source Agent Observability & Evals for AI Platform teams. Trace, test, route and govern every agent & coding assistant in the company. | 86.7 | 4.9k | TypeScript |
| 4 | OpenLLMetry A Listed Open-source observability for your GenAI or LLM application, based on OpenTelemetry | 86.6 | 7.5k | Python |
| 5 | Weights & Biases Weave A Listed Weave is a toolkit for developing AI-powered applications, built by Weights & Biases. | 84.6 | 1.1k | Python |
| 6 | Langfuse B Listed 🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform. | 79.2 | 35.8k | TypeScript |
| 7 | Opik B Listed Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards. | 78.6 | 22.6k | Python |
| 8 | Latitude B Listed Open-source observability for AI agents. Find where your agents fail, dispatch your coding agent to fix it, and verify the fix against real traces. | 77.0 | 4.7k | TypeScript |
| 9 | Evidently B Listed Evidently is an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics. | 76.1 | 8.0k | Jupyter Notebook |
| 10 | Arize Phoenix B Listed AI Observability & Evaluation | 76.1 | 11.8k | Python |
| 11 | DeepEval B Listed The LLM Evaluation Framework | 69.5 | 18.8k | Python |
| 12 | Agenta B Listed Agenta is a workspace where you and your team build agents and automations. | 68.5 | 4.8k | TypeScript |
| 13 | Giskard B Listed 🐢 Open-Source Evaluation & Testing library for LLM Agents | 67.1 | 5.9k | Python |
| 14 | Holmesgpt B Listed SRE Agent - CNCF Sandbox Project | 65.5 | 3.6k | Python |
| 15 | Helicone C Listed 🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓 | 62.3 | 6.2k | TypeScript |
| 16 | Traceroot C Listed TraceRoot - Open Source AI Agent Improvement Platform YC S25 | 61.3 | 797 | TypeScript |
| 17 | OpenLIT C Listed OpenLIT is the open-source agent harness engineering platform: trace, evaluate, guard, and improve everything around the model in your AI agents, on OTEL. | 59.8 | 2.8k | TypeScript |
| 18 | agentacct C Listed See what your coding agents did and what it cost. Breaks each task down into work steps — tools used, files changed, tests run, time and tokens spent. Local-first dashboard for Claude Code, Codex, OpenCode, and more. No login, no telemetry. | 55.4 | 766 | Python |
| 19 | Ragas C Listed Supercharge Your LLM Application Evaluations 🚀 | 53.7 | 16.0k | Python |
| 20 | AgentOps C Listed Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI | 53.2 | 5.9k | Python |
| 21 | TruLens D Listed Evaluation and Tracking for LLM Experiments and AI Agents | 47.1 | 3.6k | Python |
| 22 | SimWorld D Listed SimWorld: An Open-ended Realistic Simulator for Autonomous Agents in Physical and Social Worlds | 40.9 | 796 | Python |
| 23 | AgentBench D Listed A Comprehensive Benchmark to Evaluate LLMs as Agents (ICLR'24) | 37.3 | 3.8k | Python |
| 24 | Webarena D Listed Code repo for "WebArena: A Realistic Web Environment for Building Autonomous Agents" | 37.0 | 1.6k | Python |
| 25 | cost-xray D Listed See what Claude Code and Codex actually send to the API — and what each part costs. | 23.8 | 4.7k | Python |
| 26 | LLM Space D Listed A desktop app to prototype agent ideas, inspect every harness step, replay failures, and evaluate performance, all in one place. Local-first, cloud-ready for managed agents. | 21.8 | 2.0k | TypeScript |
Grade = trust band: A ≥ 80, B ≥ 65, C ≥ 50, D < 50 — methodology
About Observability & Evaluation
HVTracker independently evaluates 26 observability & evaluation using daily signals from GitHub, package registries, and security databases. The category is open-source-first, with a limited set of public GitHub-hosted proprietary or source-available comparators when they are important ecosystem reference points. Each agent is scored on activity, adoption, transparency, safety, and identity. The top-ranked observability & evaluation is Promptfoo with a trust score of 92.1/100 (Grade A). Other leading projects include MLflow and LangWatch.