FreeLLMAPI-Extended

One OpenAI-compatible endpoint in front of 150+ free LLMs — with health-aware routing, automatic fallback, and a full analytics dashboard.

Self-hosted, open-source LLM gateway & aggregator. Route chat, vision, image generation, embeddings, audio (STT/TTS), and reranking to 15+ free providers through a single OpenAI-compatible API — with intelligent failover so your app never goes down when one provider rate-limits.

🌍 Read this in your language: English · Türkçe · 中文 · 日本語 · 한국어 · Español · Português · Русский

📖 What is FreeLLMAPI-Extended?

FreeLLMAPI-Extended is a free, self-hosted LLM API gateway. It exposes a single OpenAI-compatible REST endpoint and transparently routes every request to the best available free model across 15+ providers (Google Gemini, Groq, Cerebras, Cloudflare Workers AI, Mistral, OpenRouter, GitHub Models, Cohere, SambaNova, NVIDIA NIM, Z.ai, and more).

When a provider rate-limits, errors, or goes down, the gateway automatically cascades to the next healthy model — your application keeps working with zero code changes. Point any OpenAI SDK at your gateway URL and you instantly get free, multi-provider, fault-tolerant inference.

Drop-in replacement for the OpenAI API. Change one base URL — keep your existing code.

✨ Features

6 modalities · 15+ providers · 150+ free models · 1 endpoint.

🏗️ Architecture

flowchart LR
    A[Your App / OpenAI SDK] -->|OpenAI-compatible /v1| B(FreeLLMAPI-Extended Gateway)
    B --> C{Health-aware Router}
    C -->|primary| D[Best healthy model]
    C -.->|fallback cascade| E[Next model / provider]
    D --> F[(150+ free models<br/>15+ providers)]
    E --> F
    B --> G[(SQLite<br/>encrypted keys · usage · cooldowns)]
    B --> H[Analytics Dashboard<br/>React]

• Backend: Node.js + TypeScript + Express, better-sqlite3 (zero external DB).
• Frontend: React analytics & key-management dashboard.
• Storage: SQLite — provider keys encrypted with AES-256-GCM.
• Routing: per-request cascade with persistent, classified cooldowns (survives restarts).

🚀 Quick Start

🐳 Docker (recommended)

# 1. Set an encryption key
echo "ENCRYPTION_KEY=$(node -e "console.log(require('crypto').randomBytes(32).toString('hex'))")" > .env

# 2. Run
docker compose up -d

The gateway is live at http://localhost:3001. Or pull the prebuilt image from GHCR:

docker run -d -p 3001:3001 \
  -e ENCRYPTION_KEY=$(openssl rand -hex 32) \
  -v "$(pwd)/data:/app/server/data" \
  ghcr.io/seyhmuskaya/freellmapi-extended:latest

🛠️ From source

# 1. Clone
git clone https://github.com/SeyhmusKaya/freellmapi-extended.git
cd freellmapi-extended

# 2. Install
npm install

# 3. Configure
cp .env.example .env
# Generate an encryption key:
node -e "console.log(require('crypto').randomBytes(32).toString('hex'))"
# Paste it into .env as ENCRYPTION_KEY=...

# 4. Run (server + dashboard)
npm run dev

Open the dashboard, add a free provider key (or use keyless providers), and you're live. See .env.example for all configuration options.

🔌 API Usage

Point any OpenAI SDK at your gateway. Leave the model field empty to auto-route to the best available model.

Python (OpenAI SDK)

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:3001/v1",   # your gateway
    api_key="YOUR_GATEWAY_KEY",
)

resp = client.chat.completions.create(
    model="",  # empty = auto-route across all free providers
    messages=[{"role": "user", "content": "Explain quantum computing in one sentence."}],
)
print(resp.choices[0].message.content)

cURL

curl http://localhost:3001/v1/chat/completions \
  -H "Authorization: Bearer YOUR_GATEWAY_KEY" \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"Hello!"}]}'

Vision (image + text)

{
  "messages": [{
    "role": "user",
    "content": [
      {"type": "text", "text": "What is in this image?"},
      {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,..."}}
    ]
  }]
}

Response headers expose the routing decision: X-Routed-Via: groq/llama-4-scout and X-Fallback-Attempts: 0.

🧠 Intelligent Routing

What makes FreeLLMAPI-Extended different from a simple proxy:

• Measured health, not guesses. The fallback chain is continuously re-ranked from the real 7-day success rate and latency of each model. A model that starts failing sinks automatically; a fast, reliable one rises.
• Classified cooldowns. Errors are bucketed (per-minute rate limit, per-day quota, dead route, invalid key) and each gets the right cooldown — a daily quota waits until UTC midnight, a transient burst waits seconds.
• Cascade-on-everything. 404 / 429 / 5xx / timeout / provider-specific 400s all trigger a skip-and-continue to the next model, so a single quirky endpoint never sinks a request.
• Keyless overflow. Anonymous providers act as last-resort capacity, so you keep serving even when every keyed provider is rate-limited.
• Per-end-user spend limits. Attribute cost to your own end users and cap daily/weekly/monthly spend.

🌐 Supported Providers

Text chat, vision, image generation, embeddings, audio (STT/TTS), and reranking across:

Google Gemini · Groq · Cerebras · Cloudflare Workers AI · Mistral · OpenRouter · GitHub Models · Cohere · SambaNova · NVIDIA NIM · Z.ai (Zhipu) · Pollinations (keyless) · Kilo Gateway (keyless) · AI21 · Reka — and an easy path to add any OpenAI-compatible provider.

Free-tier limits, model lists, and per-provider notes are documented in docs/FREE-PROVIDERS-RESEARCH.md.

📊 Dashboard

A built-in React dashboard for keys, routing, and analytics:

• Analytics — request volume, real success rate, latency, token usage, cost estimates, cascade retries, per-API-key breakdown.
• Keys — add/rotate/disable provider keys (encrypted at rest) and issue per-project consumer keys.
• Fallback — view and reorder the routing chain, or sort by measured quality.
• Playground — test models directly from the browser.

📚 Documentation

⚖️ How it compares

Looking for a self-hosted LiteLLM / OpenRouter alternative focused on free models with a dashboard? That's exactly what FreeLLMAPI-Extended is.

❓ FAQ

Is it really free? Yes — it aggregates the free tiers of many providers. You supply free API keys (or use keyless providers). The gateway itself is MIT-licensed and self-hosted.

Is it OpenAI-compatible? Yes. It implements the OpenAI Chat Completions, Embeddings, Images, Audio, and Batch shapes. Most apps only need to change the base URL.

What happens when a provider is rate-limited or down? The request automatically cascades to the next healthy model/provider. The caller never sees the failure — just a slightly different X-Routed-Via header.

Do I need a database server? No. It uses embedded SQLite (better-sqlite3). Provider keys are encrypted with AES-256-GCM.

Can I add my own provider? Yes — any OpenAI-compatible endpoint can be registered with a base URL.

How is this different from a plain proxy? Health-aware re-ranking, classified adaptive cooldowns, per-request cascade, keyless overflow, batch processing, per-end-user spend limits, and a full analytics dashboard.

🙏 Credits & Attribution

FreeLLMAPI-Extended is built on and inspired by the excellent open-source work of tashfeenahmed/freellmapi by @tashfeenahmed — huge thanks for the original foundation that made this possible.

What FreeLLMAPI-Extended adds on top of upstream

• 6 modalities — vision, image generation/editing (inpaint/outpaint), embeddings, audio (STT/TTS), and reranking, beyond text chat.
• Health-aware routing — the fallback chain is re-ranked from each model's measured 7-day success rate + latency, not just static specs.
• Batch API — OpenAI-style async batch processing with HMAC-signed webhooks, retries, and NDJSON results.
• Per-end-user spend — attribute cost to your own end users and enforce daily/weekly/monthly limits.
• Keyless providers (Pollinations, Kilo) — zero-key overflow capacity.
• Model aliases — fixed, reorder-proof routing chains for specific workloads.
• Secret/PII redaction on stored errors, request-level analytics (real success vs cascade retries), and a redesigned dashboard.

Licensed under MIT (same as upstream) — see LICENSE.

🤝 Contributing

Issues and pull requests are welcome. Whether it's a new free provider, a routing improvement, a bug fix, or docs — contributions of any size help.

FreeLLMAPI-Extended — free OpenAI-compatible LLM gateway · multi-provider AI API aggregator · self-hosted LLM router with automatic fallback.

⭐ If this project helps you, please star it to support development.

Keywords: free LLM API, OpenAI-compatible gateway, LLM aggregator, multi-provider AI router, free GPT API alternative, self-hosted AI gateway, LLM fallback, Gemini Groq Cerebras Cloudflare free API, AI proxy, free embeddings API, free image generation API.