updated 2026-08-19 05:02 UTC

Promptfoo

promptfoo/promptfoo

Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, C

Observability & Evaluation TypeScript Grade A Listed MIT
88.7/100
Rank #19 of 1308
Compare Promptfoo

How does it stack up against its Observability & Evaluation neighbours?

Pick any agent to compare →

Strong public trust posture, backed by multiple independent signals.

Open compare tool Suggest correction
Listing state
Listed
Evidence coverage
Grade A · 4/5 signals
Last push
2026-08-18 · 1d ago
Recent change
No recent signal change

Is Promptfoo safe? Promptfoo scores 88.7/100 (Grade A), ranked #19 of 1308 tracked open-source AI agent projects, on evidence coverage A (4 of 5 independent signal types). The public evidence: its packages ship with cryptographic provenance; OSSF Scorecard rates its supply-chain practices 7.5/10; recent commits are unsigned; last pushed 2026-08-18. Every point is earned from checkable signals — never paid placement. How scoring works →

Ranked neighbours in Observability & Evaluation

#1 MLflow — 89.8 (A) → #3 Weights & Biases Weave — 86.5 (A) → #4 OpenLLMetry — 84.9 (A) → #5 Langfuse — 78.1 (B) →

Quick Trust Read

What Would Improve It
Increase the share of verified signed commits for clearer maintainer identity.
Recent Changes
2026-08-11
Scorecard Added
OSSF Scorecard: 7.6/10
Maintainer Checklist
Increase signed commits Raise the share of verified-signed commits to make maintainer identity and release history easier to trust.
Compare with MLflow Get email alerts for Promptfoo View Observability & Evaluation Report missing data
91.8
Activity sub-score · out of 100
#2
In Observability & Evaluation

How to read this: HVTrust (0–100) weighs supply-chain signals (provenance, OSSF Scorecard, signed commits, open license) alongside real-world adoption. Grade A reflects the trust score band: A ≥ 80, B ≥ 65, C ≥ 50, D < 50. Evidence coverage A is separate — it grades how many independent signal types back the score (4 of 5), so a high score on thin evidence stays visible. Full methodology →

Signals refreshed 2026-08-19 05:02 UTC · Repo last pushed yesterday

Rank Trend

2026-08-11 2026-08-19

Activity & Reach

Stars
24.4k
Forks
2.2k
Last Push
2026-08-18
yesterday
Commits (4 wk)
175
Downloads (7d)
384,010
npm
HN mentions (30d)
6
Open Issues
111
Rank Change
=
was #19

Analysis

HVTrust Dimensions vs Observability & Evaluation

88.7 / 100 · 100.0% confidence

Promptfoo Observability & Evaluation average (23 agents)

Safety / Integrity50% OSSF Scorecard · 30% provenance · 20% signed commits
16.9 / 25
5.9 above avg 11.0
Identity / Provenance60% listing status · 40% build provenance
18.0 / 18
5.0 above avg 13.0
Transparency50% declared license · 50% OSSF Scorecard
14.9 / 17
3.0 above avg 11.9
Maintenance60% last-push freshness · 40% commit activity
19.9 / 20
4.8 above avg 15.1
AdoptionLog-scaled stars · package downloads
18.0 / 20
4.1 above avg 13.9

Activity Inputs

91.8 / 100
StarsRepository reach
26.3 / 30
FreshnessLast push recency
24.9 / 25
ActivityRecent commits
25 / 25
CommunityFork signal
15.5 / 20

Supply Chain Trust

Package Provenance
Verified
npm attestation
OSSF Scorecard
7.5 / 10
OpenSSF Scorecard · scanned Aug 18, 2026
Signed Commits
—
Unable to query
Binary-Artifacts 10
Branch-Protection 5
CI-Tests 10
CII-Best-Practices 0
Code-Review 6
Contributors 10
Dangerous-Workflow 10
Dependency-Update-Tool 10
Fuzzing 0
License 10
Maintained 10
Packaging 10
Pinned-Dependencies 6
SAST 9
Security-Policy 10
Signed-Releases -1
Token-Permissions 10
Vulnerabilities 0

Is Promptfoo safe?

Public supply-chain signals for Promptfoo are strong: it has multiple independent trust indicators in place. This does not replace your own security review, but Promptfoo carries less obvious unverified-evidence risk than projects with thin signals.
Does Promptfoo publish package provenance?
Yes. Promptfoo's package releases carry build provenance attestations, which cryptographically link the published package back to its source repository and CI workflow.
Does Promptfoo have an OpenSSF Scorecard?
Promptfoo has an OpenSSF Scorecard score of 7.5/10. The Scorecard checks for branch protection, signed releases, dependency updates, fuzzing, code review, and other supply-chain hygiene items. See the full check breakdown on this page.
Is Promptfoo actively maintained?
Actively maintained. The repository was pushed to within the last 1 day(s).
What license does Promptfoo use?
Promptfoo ships under MIT. A declared, OSI-approved license is one of the transparency signals HVTrust scores.

Not a safety endorsement. HVTracker describes what public signals show, not whether a project is safe for your use case. Run your own security review before adopting in production.

Compare Promptfoo head-to-head

Promptfoo vs MLflow → Promptfoo vs Weights & Biases Weave → Promptfoo vs OpenLLMetry →

AI agent surface

MCP, providers, tool surface
Scored in HVTrust

These runtime-trust fields — detected from public repo docs and manifests — contribute a bounded adjustment to this project's HVTrust score alongside supply-chain evidence. The exact values each field can add or subtract are documented in the methodology → Compare this surface across every listed agent in the capability matrix →

MCP Server Support
high confidence
Implemented
Promptfoo appears to expose MCP server capabilities.
Detailed evidence is not shown in the public view.
External Service Dependencies
high confidence
2 detected
Public provider/service dependencies detected.
Credential signal: API keys or service config markers documented.
Tool / Plugin Surface
high confidence
Declared
Declared plugin/integration surface detected.
  • browser
  • code
  • shell
Detailed evidence is not shown in the public view.
Package Provenance Drift
high confidence
Match
Published package metadata matches the tracked repo
Detailed evidence is not shown in the public view.
  • MCP signal live
  • External deps live
  • Tool / plugin surface live
  • Package provenance drift live
How this surface has changed

Detected changes to Promptfoo's runtime surface and supply-chain posture, from daily public-signal snapshots. A change here means our detectors see something different — a genuinely changed capability, or better evidence of an existing one.

2026-06-06
Provider Added
Runtime surface grew — new detected provider dependency: Google Gemini

Maintain Promptfoo?

For maintainers

HVTrust scores Promptfoo from public signals only — we never contact maintainers first. If a signal is wrong, stale, or missing (provenance you publish, a Scorecard you run, signed releases), tell us and we'll review it. Corrections are public and tracked on GitHub.

Submit a correction → Add the trust badge

Reputation Timeline

Signal history
HVTrust 3Listed 1Score 1Surface 1Rank 1Scorecard 1
2026-08-11
Scorecard Added
OSSF Scorecard: 7.6/10
2026-06-20
Rank Moved
Rank dropped 17 spots (#49 → #66)
2026-06-20
HVTrust Changed
HVTrust down 5.0pts (76.9 → 71.9)
2026-06-06
Provider Added
Runtime surface grew — new detected provider dependency: Google Gemini
2026-05-29
HVTrust Changed
HVTrust up 10.0pts (63.8 → 73.8)
2026-05-28
Activity Score Changed
Activity score up 25pts (66 → 91)
2026-05-27
HVTrust Changed
HVTrust up 14.9pts (44.7 → 59.6)
2026-05-25
Newly Listed
First tracked at rank #64
GitHub → npm: promptfoo

Embed Badge Badge guide for maintainers →

For maintainers
HVTrust 88.7 Grade A
Markdown:
[![HVTrust](https://hvtracker.net/badge/promptfoo.svg)](https://hvtracker.net/agents/promptfoo)
HTML:
<a href="https://hvtracker.net/agents/promptfoo"><img src="https://hvtracker.net/badge/promptfoo.svg" alt="HVTrust"></a>

Other agents in Observability & Evaluation

Data sources
GitHub REST API (repo, commits, stars, forks, license) · npm Registry (downloads, provenance) · OpenSSF Scorecard CLI · Algolia HN Search API
Each agent's signals refresh once daily across 6 staggered batches. Methodology v4.3 · Raw JSON