LLM Gateways & Infra
14 agents tracked · ranked by trust score · Updated 2026-10-14 00:09 UTC · the three strongest right now:
| # | Agent | Trust | Stars | Language |
|---|---|---|---|---|
| 1 | Bifrost A Listed Fastest enterprise AI gateway (50x faster than LiteLLM) with adaptive load balancer, cluster mode, guardrails, 1000+ models support & <100 µs overhead at 5k RPS. | 89.7 | 8.8k | Go |
| 2 | vLLM A Listed A high-throughput and memory-efficient inference and serving engine for LLMs | 81.0 | 93.7k | Python |
| 3 | LiteLLM B Listed The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM] | 78.3 | 61.2k | Python |
| 4 | Headroom B Listed Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server. | 76.6 | 75.3k | Python |
| 5 | Ollama B Listed Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models. | 66.2 | 183.1k | Go |
| 6 | Instructor C Listed structured outputs for llms | 64.3 | 14.0k | Python |
| 7 | Jan C Listed Jan is an open source alternative to ChatGPT that runs 100% offline on your computer. | 64.2 | 44.9k | Rust |
| 8 | LLM Gateway C Listed Route, manage, and analyze your LLM requests across multiple providers with a unified API interface. | 63.7 | 1.7k | TypeScript |
| 9 | ClawRouter C Listed The agent-native LLM router for autonomous agents. Every frontier model behind one wallet, <1ms local routing, USDC payments on Base & Solana via x402. | 58.7 | 6.6k | TypeScript |
| 10 | claude-code-router C Listed One local control plane for every AI agent: route across models, fuse new capabilities, orchestrate tools, and stay fully in control. | 57.8 | 37.7k | TypeScript |
| 11 | Portkey AI Gateway C Listed A blazing fast AI Gateway with integrated guardrails. Route to 1,600+ LLMs, 50+ AI Guardrails with 1 fast & friendly API. | 52.3 | 13.2k | TypeScript |
| 12 | paritok-4b-v1 D Listed Non-destructive compression gateway for AI coding agents. Cuts token bills 25% on turn 1 to past 85% in long or saturated sessions, and fits ~3× more turns in the same context window. Powered by our open-source code-native 4B model. Drop-in for Claude Code, Cursor, Codex, OpenHands, and any… | 45.8 | 1.5k | Python |
| 13 | Chat Nio D Listed 🚀 Next Gen Multi-tenant AI One-Stop Solution. Builtin Admin & Billing System. Enterprise-Grade Unified LLM Gateway Support for 200+ Models And 35+ Providers, Load Balacing w/ Priority-base Routing, Cost Management, Chat Share, Cloud Sync, Credit/Subscription Billing, All File Parsing, Web Search… | 37.7 | 9.3k | TypeScript |
| 14 | LocalAI D Listed LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required. | 37.5 | 49.5k | Go |
Grade = trust band: A ≥ 80, B ≥ 65, C ≥ 50, D < 50 — methodology
About LLM Gateways & Infra
HVTracker independently evaluates 14 llm gateways & infra using daily signals from GitHub, package registries, and security databases. The category is open-source-first, with a limited set of public GitHub-hosted proprietary or source-available comparators when they are important ecosystem reference points. Each agent is scored on activity, adoption, transparency, safety, and identity. The top-ranked llm gateways & infra is Bifrost with a trust score of 89.7/100 (Grade A). Other leading projects include vLLM and LiteLLM.