Awesome AI Gateway

Pick one in ~10 seconds, understand why, and check every number yourself. A decision tree over 160+ gateways, a handbook that explains how they actually work (and fail), and a reproducible cost benchmark plus dated evidence behind every claim — including what we exclude and why. CC0, no vendor money, no affiliate links.

💸 Built the hard way: I burned $788 on AI coding in a single day — one flagship model ate 78% of it, just because I'd defaulted everything to the priciest option. So I mapped the whole gateway landscape. → full story in Why this exists below · the full comparison

Languages: English · 简体中文

Contents

• Which gateway should I use
  • ✅ Why trust this list
• 🌍 State of the landscape (2026)
• ⚡ 10-second answers
• 📖 Learn: how AI gateways work
• 🔥 Top gateways (by stars)
  • 🩺 Maintenance signal — what the star count hides
• 💰 Cost-first: cheapest multi-model access
  • 🆓 Free tiers that still work — the verified-limits table
• 🔓 Self-hosted open source
• 🏢 Enterprise & compliance
• 🌐 First-party gateways (cloud & model vendors)
• 🇨🇳 China ecosystem
• 🤖 MCP & agent gateways
• 🔧 More by capability (cross-cutting)
  • 🧠 Smart routing & model selection
  • 💾 Prompt caching through a gateway — the money question
  • 📊 Observability & cost tracking
  • 🚢 Kubernetes-native & inference infra
• Quick comparison
• The requirements map
• How to choose safely
  • 🔒 Who sees your prompts? — the data-retention matrix
  • 🔐 Supply-chain security — who signs their releases, and what actually got hacked
  • 🧰 Companion tools — verify what you picked
  • Community relay watch-list
• 📊 Latest evaluations
• 📰 What's new
• 🚀 Recent releases (auto-updated)
• 📚 Essential reading
• Guides & comparisons
• FAQ
• Glossary
• Why this exists
• 🔌 Use the data — it's an API
• 🔗 Related lists
• Star history

Which gateway should I use

⚡ Fast answer — one sane default per need (alternatives in each linked section):

Do you want to self-host?
│
├─ NO — hosted, minimal ops
│   ├─ Cheapest access to many models ──────────▶ OpenRouter · Vercel AI Gateway (0% markup)
│   ├─ Free control plane over your own keys ───▶ Cloudflare AI Gateway
│   ├─ EU data residency matters ───────────────▶ Requesty · Eden AI · nexos.ai
│   └─ Already on one cloud ────────────────────▶ AWS Bedrock · Azure APIM · Vertex AI
│
└─ YES — self-hosted / open source
    ├─ Python stack, broadest features ─────────▶ LiteLLM
    ├─ Raw performance (Go/Rust/TS) ────────────▶ Bifrost (0.62 ms measured) · Portkey Gateway (2.65 ms; fidelity 1/3)
    ├─ Built-in evals + observability ──────────▶ LiteLLM · Bifrost · Helicone (maintenance mode)
    ├─ Multi-user keys + budgets + admin UI ────▶ LiteLLM · new-api   (family / small team)
    ├─ CN models + CNY billing ─────────────────▶ new-api · one-api · GPT-Load
    ├─ Enterprise K8s, audit, guardrails ───────▶ Kong · Higress · APISIX · Envoy AI Gateway
    └─ Governing AI agents & MCP traffic ───────▶ agentgateway · Lunar.dev

✅ Why trust this list

• ✅ Independent — no vendor money, no affiliate links, CC0. Unlike affiliate-driven relay "rankings," nobody pays to appear here.
• ✅ Reproducible, not asserted. Every cost cell is computed from open pricing data by a unit-tested script; stars refresh daily via CI.
• ✅ Honest about risk. We disclose CVEs, label archived/stale projects, and exclude gray-market relays — with the research to back it.

Why this matters: the same task can cost 100× more depending on the model behind your gateway. An AI gateway sits between your code and LLM providers — one endpoint, one key, many models — handling routing, failover, caching, rate limits, cost tracking and guardrails, so you change a base_url instead of rewriting your app. Pick the gateway here, then the evaluation set shows which model to route to.

⭐ Found this useful? Star it — that's how the next engineer choosing a gateway finds it. CC0, no signup, no tracking, no vendor money.

🌍 State of the landscape (2026)

The standalone AI gateway is being squeezed from both ends. Underneath, hyperscalers now ship routing and spend caps natively; above, the coding-agent wave produced a faster-growing class of gateway that most round-ups miss. The part that consolidated is the independent middle — a company whose entire product was the gateway.

Not a collapse — the same year, LiteLLM ⭐ 56.5k kept compounding, Bifrost ⭐ 7.4k won on measured overhead, Envoy AI Gateway reached v1.0 (Jun 23) as the first CNCF-backed production-stable option, and the coding-agent tier plus token-compression proxies became the fastest-growing categories in this list.

Three things that changes for you: check the last commit before the feature matrix; prefer code you can keep over a control plane you cannot; and if your traffic already lives in one cloud, that cloud's native gateway is now a real answer.

📉 The full picture — the consolidation evidence, the six-check evaluation standard, and three dated predictions with their disproof conditions: LANDSCAPE.md

⚡ 10-second answers

The questions people actually ask (sourced from real threads) — answered first:

📖 Learn: how AI gateways work

The list tells you which gateway; these chapters teach you how they work and why they fail — same evidence bar (dated, sourced, reproducible). Chapter map + reading order: HANDBOOK.md.

• 📕 The compatibility surface - Why gateways break Claude Code: three wire protocols field-by-field, five translation failure modes each anchored to a verified GitHub issue, the measured fidelity results, and a 10-minute self-test for your own gateway.
• 📗 Routing & model selection: the research landscape - Cost-aware cascades, learned routers, ensembling and self-routing — with the honest counter-evidence on when routing doesn't pay.
• 📘 Observability: what to measure and why - The OTel GenAI conventions, the metric tiers that separate instrumented from blind, and silent model drift.
• 📙 Anatomy of an AI gateway - The request lifecycle read from seven gateways' source at pinned commits: where the cache, the budget check and the retry boundary actually sit, why metering rarely survives a crash, and the six conditions under which you shouldn't run a gateway at all.
• 📕 Failover & reliability - What actually happens when a provider fails mid-request: only one of six gateways retries by default, what a client sees when the stream dies after the first token, whether a retry can be billed twice, and three providers' incompatible 429 contracts.
• 📗 Caching economics - Reads are 0.1× but writes are 1.25–2×, so caching loses money below a 21.7% hit rate — the formula, the four providers' non-portable rules, semantic-cache false hits with receipts, and a 30-second self-test.
• 📘 Virtual keys, budgets & metering - How a gateway counts your money: what a virtual key really scopes, pre-spend reservation vs post-hoc metering under concurrency, how streamed and reasoning and cached tokens get mis-counted, and what spend disappears when the process dies.
• 📕 MCP & agent gateways - Why agent traffic is not completion traffic: six properties change at once, MCP went stateless on 2026-07-28, tool-level authorization and secret brokering, and the DNS-rebinding CVE class that hit five codebases.
• ✅ All eight chapters are live — the map and reading order live in HANDBOOK.md, linked above.

🔥 Top gateways (by stars)

Every project in the list that clears the last row here, sorted by stars — a fast orientation. Archived projects are held back (TensorZero went read-only in June 2026) so the table stays things you can actually adopt today. Stars auto-refresh daily; full context (features, license, caveats) is in each linked section. ⚠️ flags a project whose coding-subscription / account-pool routing can carry provider-ToS or account-ban risk.

Hosted leaders (SaaS, no GitHub stars): OpenRouter (~340 models, ~5.5% fee) · Vercel AI Gateway & Cloudflare AI Gateway (0% markup) → Cost-first.

Stars measure popularity, not fitness for your need — that's what the category sections and the evidence-based scorecard are for.

🩺 Maintenance signal — what the star count hides

Stars are cumulative; they never go down when a project stops shipping. Every tracked repo below is either archived or has taken no commit in six months. Refreshed daily from the GitHub API, same pass as the star counts — so this cannot go stale while the stars stay current.

A quiet repo is not automatically a bad pick — a stable proxy may simply be finished, and a listed project may have moved its work elsewhere. It is a question to ask before you depend on it, not a verdict.

💰 Cost-first: cheapest multi-model access

🎯 Pain point: "I want many models for the least money and zero ops."

Where the line is: stolen-quota / reverse-engineered access is excluded from this list outright; transparent resellers are listed — but flagged New & unverified and folded below until an independent fidelity check passes.

Start here — the options with a track record:

• OpenRouter - The dominant model marketplace: ~340 models (live API count) behind one OpenAI-compatible API, pay-as-you-go with automatic failover; ~5.5% fee when buying credits. $113M Series B (May 2026), ~8M users.
• Vercel AI Gateway - Hundreds of models at provider list price (0% markup), $5/month free credits, zero-data-retention option; pairs naturally with the AI SDK.
• Cloudflare AI Gateway - Free control plane in front of your own provider keys: caching, dynamic routing, unified billing, and dollar-denominated spend limits (2026 beta).
• Requesty - EU-friendly OpenRouter alternative: 400+ models, sub-20ms failover, ~5% markup.
• Eden AI - Unified API for 500+ models plus vision/OCR/speech; EU-based, ~5.5% platform fee.
• Helicone AI Gateway (cloud) - Passthrough billing at 0% markup with observability bundled. Acquired by Mintlify (2026-03) and in maintenance mode — fine to use today, but weigh the roadmap risk.
• GPT-Load ⭐ 6.3k - High-performance Go proxy that rotates pools of API keys across channels to maximize quota usage.
• freellmapi ⭐ 18.7k - OpenAI-compatible proxy (MIT) that stacks the free tiers of 28 LLM providers behind one /v1 endpoint — smart routing, automatic failover, per-key quota tracking, encrypted key storage. ⚠️ Stacking provider free quotas behind one endpoint can carry provider-ToS / account-ban risk — the repo itself says "personal experimentation only" — and the operator sells a paid live-catalog subscription ($19/yr; the router itself stays MIT, self-hosted, on your own keys).
• AIMLAPI - One OpenAI/Anthropic-compatible endpoint fronting 400+ models (chat, image, video, audio, embeddings); prepaid, OpenRouter-style aggregator.
• Novita AI - Unified API to 200+ open-source models (DeepSeek/Qwen/Llama…) with load balancing, autoscaling and failover; also a GPU cloud.
• Glama Gateway - OpenAI-compatible gateway to 100+ models with consolidated billing, caching and logging (OSS core glama-ai/lightport).

• Loop Gateway - OpenAI-compatible proxy that meters every request in Bitcoin sats instead of dollars. 311 models via OpenRouter at a 15% markup. No accounts, no email, no card; top up over Lightning, get a bearer token. Three auth rails (prepaid bearer, L402, Cashu). Hosted at api.loopxxi.com. New & unverified (anonymous; its public GitHub repo has since been removed, so treat it as a closed hosted relay) — it resells frontier models through the operator's own OpenRouter account at a 15% markup, and account-less + crypto-prepaid means no recourse if it swaps models or vanishes; confirm fidelity with canary_check.py and only top up what you can afford to lose.
• nullsink (repo) - Account-less, metered proxy for frontier-model APIs, paid in Monero or Bitcoin. No accounts, no email, no card; mint a bearer token, prepay on-chain, and point the official SDKs at one base URL. ~10% markup taken once at top-up; no IP logging, no request logs; payment and token kept unlinkable. Self-hostable single binary (TypeScript/Bun, AGPL-3.0), live at nullsink.is. New & unverified (repo created 2026-06, ⭐ 12) — account-less + crypto-prepaid + no logs means no recourse if it swaps models or vanishes; confirm fidelity with canary_check.py and only top up what you can afford to lose.
• RunAPI - Hosted OpenAI-compatible relay (https://runapi.ai/v1) spanning LLM + media-generation (image, video, music/audio) behind one developer account; ships an active CLI and SDKs (runapi-ai) though the gateway itself is closed/hosted. New & unverified (self-submitted) — confirm model fidelity (e.g. with canary_check.py) before relying on it in production.
• TeamoRouter - Hosted SaaS gateway speaking OpenAI, Anthropic Messages and Gemini APIs over 500+ providers, with quality/latency/cost-aware "agentic routing" and a channel-dilution check that reroutes when a provider's output quality degrades; native Claude Code / Codex support, MCP, pay-as-you-go plus plans (Alipay/WeChat). New & unverified (the routing / quality-detection behavior is the operator's own description) — confirm model fidelity (e.g. with canary_check.py) before relying on it in production.
• KeepRouter - OpenAI- and Anthropic-compatible gateway: one key fronts 50+ models (Claude, GPT, Gemini, Mistral, Qwen, Kimi, GLM, DeepSeek, MiMo, MiniMax). Native /v1/messages means it works with Claude Code and the Anthropic SDK, not just the OpenAI SDK. Prepaid pay-as-you-go, billed at cost with 0% token markup (top-up fee 8%+$0.35); one genuinely free $0 model plus trial credit for new accounts. Bilingual EN/简体中文, Merchant of Record Paddle; not available in mainland China. New & unverified — confirm fidelity with canary_check.py before relying on it in production.
• AI快站 (aifast.club) - Operator-submitted OpenAI- and Anthropic-compatible relay aimed at the mainland-China market (Cursor / Claude Code / Codex / Dify integration docs). Its current model list and rates are published live at /api/ratio_config, with service status at /api/status — read those rather than a fixed snapshot. New & unverified (closed-source, self-submitted) — confirm model fidelity with canary_check.py before relying on it in production.
• ApiFlux - Hosted OpenAI-, Anthropic- and Gemini-compatible gateway with dedicated connection guides for Claude Code, Codex CLI and OpenCode, plus per-key usage and request logs in the console; the operator advertises all models at 85% of list price. New & unverified (self-submitted; below-list pricing makes independent fidelity checks especially important) — confirm model fidelity with canary_check.py before relying on it in production.
• NovAI - Hosted OpenAI-compatible relay fronting Chinese frontier models (DeepSeek, Qwen, GLM, Kimi, MiniMax, Doubao, Hunyuan) under one key; unlike the other Chinese-model relays here it also exposes image (Doubao Seedream) and video (Doubao Seedance) generation on the same endpoint, per-token pay-as-you-go with trial credit for new accounts. New & unverified (self-submitted) — confirm model fidelity (e.g. with canary_check.py) before relying on it in production.
• FlintAPI (repo) - Hosted OpenAI-compatible relay over Chinese LLMs (DeepSeek, Qwen, Kimi, GLM, MiniMax); the operator now describes it as a smart-routing engine that dispatches each prompt to a best-suited model rather than a plain aggregator, with free trial credit for new accounts. New and unverified (routing behavior is the operator's own description) — confirm model fidelity (e.g. with canary_check.py) before relying on it in production.
• FlowBar - Hosted OpenAI-compatible relay reselling dozens of models (GPT, Claude, Gemini, DeepSeek, Qwen, GLM, Kimi) below OpenRouter. Its differentiator is payment reach for buyers outside the US card system: Alipay/WeChat, USDT/USDC, PayPal and Apple/Google Pay through a local-methods processor. New and unverified — confirm model fidelity (e.g. with canary_check.py) before relying on it in production.
• Meshs One - Hosted OpenAI-compatible relay fronting Chinese frontier models (DeepSeek-V4, Qwen3.7-Max, MiniMax-M3) under one key, per-token pay-as-you-go (its /v1 endpoint returns a new_api_error, so it appears to run on new-api). New & unverified — closed-source and brand-new; confirm model fidelity (e.g. with canary_check.py) before relying on it in production.
• CoderPlan - Hosted OpenAI-compatible relay fronting Claude/GPT/Gemini/DeepSeek/Grok for the China market, per-token pay-as-you-go, ¥10 minimum top-up with Alipay/WeChat; Hong Kong/Singapore nodes (API base api.coderplan.ai/v1, which returns a new_api_error, so it appears to run on new-api). New and unverified — confirm model fidelity (e.g. with canary_check.py) before relying on it in production.
• lxg2it ModelRouter (repo) - Solo-built, OpenAI-compatible router over 7+ providers (Anthropic, OpenAI, Google, Cerebras, Groq, Grok, GLM) with tiered automatic fallback that selects the cheapest available model. Free tier plus a paid tier advertised at 0% markup on Anthropic (a deposit fee may apply — verify current pricing). New and unverified — the public repo is a thin, unlicensed core stub still seeing fresh commits (routing logic lives in the closed hosted service; no license file) — confirm model fidelity (e.g. with canary_check.py) before relying on it in production.
• OpenPaths (repo) - Hosted OpenAI-compatible router auto-routing across 15+ providers (OpenAI, Anthropic, Gemini, Groq, xAI, DeepSeek, Mistral) under one API, spanning chat, image, video, music, speech, embeddings and transcription. New & unverified — despite the "open source" framing the GitHub repo is a no-code, unlicensed marketing mirror (canonical code lives on the third-party Codex Infinity platform and is agent-maintained), so treat it as a closed hosted relay and