Registry › Compare › OpenLLMetry vs Promptfoo

OpenLLMetry vs Promptfoo

Promptfoo leads on trust: 92.6/100 (Grade A) against 85.9/100 (Grade A), a 6.7-point gap. Promptfoo leads on supply-chain integrity and maintenance, and rests on broader evidence.

A OpenLLMetry 85.9

Open-source observability for your GenAI or LLM application, based on OpenTelemetry

traceloop/openllmetry · #87 overall · #4 Observability & Evaluation · coverage B (3/5)

OpenLLMetry doesn't lead on any scored dimension in this pair.

A Promptfoo 92.6

Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.

promptfoo/promptfoo · #8 overall · #1 Observability & Evaluation · coverage A (4/5)

Choose Promptfoo if supply-chain integrity and maintenance matter most.

+3.6Maintenance: last push today, against 1d ago
+2.4Safety / Integrity: 100% of recent commits signed, against 82%
+2.0Adoption: 1.3M weekly downloads against 449.4k
+1.0Transparency: OSSF Scorecard 7.5 against 6.3

Where they differ

19.5

Safety / IntegrityPromptfoo +2.4

21.9

13.9

TransparencyPromptfoo +1.0

14.9

16.4

MaintenancePromptfoo +3.6

20.0

16.8

AdoptionPromptfoo +2.0

18.8

+1.3

Runtime calibrationOpenLLMetry +2.3

-1.0

1 dimension identical: Identity 18.0 · Full evidence table

An independent, evidence-based trust comparison of OpenLLMetry and Promptfoo, two Observability & Evaluation projects in the HVTracker registry. Scores come from public, checkable signals — supply-chain provenance, OSSF Scorecard, maintenance, and adoption — not popularity.

Full evidence

Signal	OpenLLMetrytraceloop/openllmetry	Promptfoopromptfoo/promptfoo
HVTrust score	85.9	92.6
Evidence grade	A	A
Coverage grade	B	A
Overall rank	#87	#8
Rank in Observability & Evaluation	#4	#1
GitHub stars	7.5k	26.8k
Last updated	1d ago	today
Build provenance	Yes	Yes
OSSF Scorecard	6.3 / 10	7.5 / 10
License	Apache-2.0	MIT
Downloads	449k/wk	1.3M/wk
Trust dimensions (points earned)
Safety / integrity / 25	19.5	21.9
Identity & provenance / 18	18.0	18.0
Transparency / 17	13.9	14.9
Maintenance / 20	16.4	20.0
Adoption / 20	16.8	18.8
Runtime capability surface (full matrix)
MCP server	Implemented	Implemented
External providers	9 — Amazon Bedrock, Anthropic, Cohere, …	3 — Amazon Bedrock, Anthropic, OpenAI
Requires API keys	No	Yes
Plugin surface	—	plugins
Provenance drift	Match	Match

Open in the live compare tool → OpenLLMetry profile Promptfoo profile More Observability & Evaluation →

How to read this: HVTrust (0–100) weighs supply-chain signals (provenance, OSSF Scorecard, signed commits, open license) alongside real-world adoption, scaled by an evidence-confidence factor. Grade bands: A ≥ 80, B ≥ 65, C ≥ 50, D < 50. Signals refresh daily. Full methodology v4.4 →