ccLoad

AI API gateway for Claude Code, Codex, Gemini, and OpenAI.

English | 简体中文

Smart routing | Automatic failover | Model-aware cooldown | Multi-URL scheduling | Protocol transforms | Live monitoring | Cost control

ccLoad removes the operational mess of running multiple AI API upstreams. It keeps Claude Code, Codex, Gemini, and OpenAI-compatible clients on one stable gateway, then handles upstream selection, failover, cooldown, protocol conversion, request visibility, and cost limits in the service instead of in every client script.

🤖 Built with Codex and GPT-5.6

During OpenAI Build Week, Codex powered by GPT-5.6 was the primary engineering agent used to:

• Trace request routing, failover, cooldown, protocol conversion, and dashboard flows across the Go backend and embedded web UI.
• Implement and review model-scoped cooldown handling for upstream 5xx, key-level 429, and model-unavailable 404 failures without unnecessarily cooling an entire channel.
• Refine the model-status and call-statistics UI, update the English and Chinese documentation, and verify the result with focused Go tests, builds, and browser walkthroughs.
• Prepare the reproducible demo and Devpost submission while keeping architecture, security, and final-review decisions under human control.

GPT-5.6 is also integrated into the product itself: ccLoad exposes GPT-5.6 through OpenAI-compatible and Codex Responses endpoints, includes Sol, Terra, and Luna model presets, calculates their standard, priority, flex, cached-token, and long-context costs, and applies routing and model-scoped cooldown decisions to them like any other configured upstream model.

The repository's AGENTS.md and CLAUDE.md provide persistent engineering constraints so Codex works against the same KISS-first review and testing rules in every session.

🎯 What ccLoad Solves

Common failure modes when you run several AI API channels:

• Manual channel switching: Different keys, validity windows, quotas, and upstream URLs quickly become hard to manage.
• Rate limits and upstream failures: 429, 502, 504, expired keys, and overloaded providers should not stop the client workflow.
• Opaque request status: Without live request visibility, long streaming requests become guesswork.
• HTTP 200 with error content: Some upstreams return a successful HTTP status while the response body is an actual error.
• Cost drift: Shared gateways need per-channel and per-token limits, not spreadsheet accounting after the bill arrives.

ccLoad handles those cases with:

• Smart routing: High-priority channels are selected first; channels at the same priority use smooth weighted round-robin.
• Automatic failover: Failed keys, models, channels, and URLs are skipped according to the classified error scope.
• Model-aware cooldown: Structured model_cooldown responses, upstream HTTP 5xx failures, key-level 429 rate limits, and model-unavailable 404 errors all cool only the actual upstream model first; other models on the same channel remain available. The channel is promoted to cooldown only after every configured model or every enabled key is cooling.
• Multi-URL scheduling: A single channel can use multiple upstream URLs, weighted by observed latency and health.
• Protocol transforms: Anthropic, OpenAI, Gemini, and Codex request/response families can be converted at the gateway.
• Live monitoring: Active requests, logs, token usage, TTFB, cost, and upstream details are visible in the web dashboard.
• Soft-error detection: HTTP 200 responses that are actually errors trigger the same failover path as regular upstream failures. Common cases include:
  • JSON responses containing {"error": {...}} structure
  • Responses with type field set to "error"
  • Explicit rate limits in SSE error events (rate_limit_exceeded / too_many_requests) are handled as 429
  • Plain text messages like "当前模型负载过高" / "Current model load too high" (load warnings)

✨ Key Features

• 🚀 High-Performance Architecture - Gin framework, 1000+ concurrent connections, high-performance caching
• 🧮 Local Token Counting - API-compliant local token estimation, <5ms response, 93%+ accuracy, supports large-scale tool scenarios
• 🎯 Smart Error Classification - Distinguishes Key/Model/Channel/Client errors, soft error detection (200 masquerading as error), SSE rate-limit errors as 429, 1308 quota handling
• 🔀 Smart Routing - Priority + smooth weighted round-robin channel selection, pre-filters cooled channels, multi-key load balancing, health-based dynamic sorting (confidence factor prevents small sample over-penalization)
• 🛡️ Failover - Key/channel failures use exponential backoff; model cooldowns (structured responses, 5xx, 429, model-unavailable 404) honor the upstream reset deadline and switch channels without cooling the whole channel
• 🔒 Race-Safe - Key selector race condition protection, startup config validation, automatic resource cleanup
• 📊 Real-time Monitoring - Built-in trend analysis, logging, and stats dashboard, Token usage stats with time range selection and per-token classification
• 🎯 Transparent Proxy - Supports Claude Code, Codex, Gemini, and OpenAI compatible APIs with smart auth detection
• 📦 Single Binary Deployment - No external dependencies, embedded SQLite included
• 🔒 Secure Authentication - Token-based admin interface and API access control
• 🏷️ Build Tags - GOTAGS support, high-performance JSON library enabled by default
• 🐳 Docker Support - Multi-arch images (amd64/arm64), automated CI/CD
• ☁️ Cloud Native - Container deployment support, GitHub Actions auto-build
• 🤗 Hugging Face - One-click deployment to Hugging Face Spaces, free hosting
• 💰 Cost Limits - Per-channel daily cost limits, per-token cost limits
• 🚦 Channel RPM Limits - Per-channel rolling 60-second request caps, 0=unlimited
• 🚧 Channel Concurrency Limits - Per-channel in-flight request caps, 0=unlimited
• 🔐 Token Restrictions - Per-token cost limits, model restrictions, channel allowlist/denylist, and concurrency caps for fine-grained access control
• ⏱️ TTFB Monitoring - Streaming request first byte time tracking for upstream latency diagnosis
• 🌐 Multi-URL Load Balancing - Multiple URLs per channel with latency-weighted random selection
• 💵 service_tier Pricing - OpenAI priority/flex/default tier multipliers for accurate cost accounting
• 🖼️ Image Tool Billing - Responses image_generation/gpt-image-2 cost accounting
• 📉 Tiered Pricing - GPT-5.4/Qwen-Plus/Gemini long-context step pricing, auto-applies lower rate at token thresholds
• 🔄 Protocol Transform - All 12 directed Anthropic/OpenAI/Gemini/Codex conversion paths for requests plus streaming and non-streaming responses, including tool, reasoning/signature, usage, and SSE normalization
• 💬 Conversational Model Testing - Channel/model/chat testing modes with image upload, reasoning level, built-in search, and chat export
• 🔍 Debug Logs - Upstream request/response raw data capture with sensitive header masking, essential for troubleshooting
• 🕐 Scheduled Checks - Background periodic channel availability probing, auto-detect failed channels
• 🔄 Auto Updates - Checks for new releases every 12 hours by default, configurable from the admin settings page
• 🧩 Custom Request Rules - Per-channel HTTP header & JSON body rewriting (remove/override/append), with auth header protection, CRLF guard, and capacity caps
• 🎛️ Log Column Customization - Show/hide table columns per preference, settings persist in browser localStorage

🏗️ Architecture Overview

graph TB
    subgraph "Client"
        A[Claude Code / Codex / Gemini / OpenAI Client]
    end

    subgraph "ccLoad Service"
        B[HTTP Proxy]
        C[Authentication + Route Dispatch]
        D[Channel Selector<br/>Priority + Smooth Weighted RR]
        E[Protocol Registry<br/>Native Bypass / Local Transform]
        F[URL Selector<br/>Explore + 1/EWMA Weighting]
        G[(Storage Factory<br/>SQLite / MySQL / PostgreSQL)]
        H[Logs + Metrics + Cost Control]

        A --> B --> C --> D --> E --> F
        D <--> G
        H <--> G
        B --> H
    end

    subgraph "Upstream Services"
        U1[Anthropic]
        U2[OpenAI-compatible]
        U3[Gemini]
        U4[Codex Responses]
    end

    F --> U1
    F --> U2
    F --> U3
    F --> U4
    U1 -. JSON / SSE .-> E
    U2 -. JSON / SSE .-> E
    U3 -. JSON / SSE .-> E
    U4 -. JSON / SSE .-> E
    E -. Client Protocol .-> B

    style B fill:#4F46E5,stroke:#000,color:#fff
    style D fill:#059669,stroke:#000,color:#fff
    style E fill:#0EA5E9,stroke:#000,color:#fff

🚀 Quick Start

Choose the deployment method that suits you best:

Method 1: Docker Deployment (Recommended)

Using pre-built images (Recommended):

# Option 1: Using docker-compose (Simplest)
curl -o docker-compose.yml https://raw.githubusercontent.com/caidaoli/ccLoad/master/docker-compose.yml
curl -o .env https://raw.githubusercontent.com/caidaoli/ccLoad/master/.env.example
# Edit .env file to set password
docker-compose up -d

# Option 2: Run image directly
docker pull ghcr.io/caidaoli/ccload:latest
docker run -d --name ccload \
  -p 8080:8080 \
  -e CCLOAD_PASS=your_secure_password \
  -v ccload_data:/app/data \
  ghcr.io/caidaoli/ccload:latest

Building from source:

# Clone project
git clone https://github.com/caidaoli/ccLoad.git
cd ccLoad

# Build and run with docker-compose
docker-compose -f docker-compose.build.yml up -d

# Or build manually
docker build -t ccload:local .
docker run -d --name ccload \
  -p 8080:8080 \
  -e CCLOAD_PASS=your_secure_password \
  -v ccload_data:/app/data \
  ccload:local

Method 2: Source Build

# Clone project
git clone https://github.com/caidaoli/ccLoad.git
cd ccLoad

# Build project (uses high-performance JSON library by default)
go build -tags sonic -o ccload .

# Or use Makefile
make build

# Run in development mode
go run -tags sonic .
# Or
make dev

Method 3: Binary Download

# Download binary for your platform from GitHub Releases
wget https://github.com/caidaoli/ccLoad/releases/latest/download/ccload-linux-amd64
chmod +x ccload-linux-amd64
./ccload-linux-amd64

Method 4: Hugging Face Spaces Deployment

Hugging Face Spaces provides free container hosting with Docker support, ideal for personal and small team use.

Deployment Steps

1. Login to Hugging Face

   Visit huggingface.co and log into your account

2. Create New Space

   • Click "New" → "Space" in the top right
   • Space name: ccload (or custom name)
   • License: MIT
   • Select the SDK: Docker
   • Visibility: Public or Private (private requires paid subscription)
   • Click "Create Space"

3. Create Dockerfile

   Create a Dockerfile in the Space repository:

   FROM ghcr.io/caidaoli/ccload:latest
   ENV TZ=Asia/Shanghai
   ENV PORT=7860
   ENV SQLITE_PATH=/tmp/ccload.db
   EXPOSE 7860
   

   Create via:

   Method A - Web Interface (Recommended):

   • Click "Files" tab on Space page
   • Click "Add file" → "Create a new file"
   • Enter Dockerfile as filename
   • Paste the content above
   • Click "Commit new file to main"

   Method B - Git Command Line:

   # Clone your Space repository
   git clone https://huggingface.co/spaces/YOUR_USERNAME/ccload
   cd ccload
   
   # Create Dockerfile
   cat > Dockerfile << 'EOF'
   FROM ghcr.io/caidaoli/ccload:latest
   ENV TZ=Asia/Shanghai
   ENV PORT=7860
   ENV SQLITE_PATH=/tmp/ccload.db
   EXPOSE 7860
   EOF
   
   # Commit and push
   git add Dockerfile
   git commit -m "Add Dockerfile for ccLoad deployment"
   git push
   

4. Configure Environment Variables (Secrets)

   In Space settings (Settings → Variables and secrets → New secret):

   Variable	Value	Required	Description
   CCLOAD_PASS	your_admin_password	✅ Required	Admin interface password
   CCLOAD_API_TOKENS	token1|production,token2|development	Optional	Pre-seed API access tokens on startup

   Note: API access tokens can be pre-seeded with CCLOAD_API_TOKENS or managed in the Web admin interface /web/tokens.html.

5. Wait for Build and Startup

   After pushing Dockerfile, Hugging Face will automatically:

   • Pull pre-built image (~30 seconds)
   • Start application container (~10 seconds)
   • Total time ~1-2 minutes (3-5x faster than source build)

6. Access Application

   After build completes, access via:

   • App URL: https://YOUR_USERNAME-ccload.hf.space
   • Admin Interface: https://YOUR_USERNAME-ccload.hf.space/web/
   • API Endpoint: https://YOUR_USERNAME-ccload.hf.space/v1/messages

   First Access Note:

   • If Space is sleeping, first access takes 20-30 seconds to wake
   • Subsequent accesses respond immediately

Hugging Face Deployment Characteristics

Advantages:

• ✅ Completely Free: Public Spaces are permanently free with CPU and storage
• ✅ Fast Deployment: Pre-built image, 1-2 minutes (3-5x faster than source build)
• ✅ Auto HTTPS: No SSL certificate configuration needed
• ✅ Auto Restart: Automatic restart after crashes
• ✅ Version Control: Git-based, easy rollback and collaboration
• ✅ Simple Maintenance: Only 5-line Dockerfile, no source code management

Limitations:

• ⚠️ Resource Limits: Free tier provides 2 CPU + 16GB RAM
• ⚠️ Sleep Policy: 48 hours without access triggers sleep, first access takes ~20-30s to wake
• ⚠️ Fixed Port: Must use port 7860
• ⚠️ Public Access: Spaces are public by default, must configure API tokens via Web admin to access /v1/* APIs (otherwise 401)

Data Persistence

Important: Hugging Face Spaces Storage Policy

Due to Hugging Face Spaces limitations (/tmp directory clears on restart), we strongly recommend using an external MySQL or PostgreSQL database for complete data persistence:

Option 1: Hybrid Storage Mode (Recommended, Best Performance)

• ✅ Ultra-fast queries: Reads go through local SQLite, avoiding remote database latency
• ✅ Restart-safe: Durable data is stored in MySQL/PostgreSQL and restored on startup
• ✅ Stats caching: Smart TTL cache reduces repetitive aggregate queries
• Configuration: Add one primary DSN (CCLOAD_MYSQL or CCLOAD_POSTGRES) plus CCLOAD_ENABLE_SQLITE_REPLICA=1 in Secrets

Dockerfile Example (Hybrid Mode):

FROM ghcr.io/caidaoli/ccload:latest
ENV TZ=Asia/Shanghai
ENV PORT=7860
# Configure in Secrets: CCLOAD_MYSQL or CCLOAD_POSTGRES, plus CCLOAD_ENABLE_SQLITE_REPLICA=1
EXPOSE 7860

Option 2: Pure External Database Mode

• ✅ Complete Persistence: Channel configs, logs, and stats all preserved
• ✅ Restart-Safe: Data stored externally, unaffected by Space restarts
• ⚠️ Database Latency: Stats page latency depends on the remote database and region
• Configuration: Add exactly one of CCLOAD_MYSQL or CCLOAD_POSTGRES in Secrets

Recommended Free MySQL Services:

• TiDB Cloud Serverless - Free 5GB storage, MySQL compatible, no connection limits, recommended first choice
• Aiven for MySQL - Free 1GB storage, multi-region support

MySQL Configuration Example (TiDB Cloud):

1. Register for TiDB Cloud account
2. Create Serverless Cluster (free)
3. Get connection info, format: user:password@tcp(host:4000)/database?tls=true
4. Add CCLOAD_MYSQL variable in Hugging Face Space Secrets
5. (Optional) Enable Hybrid Mode: Add CCLOAD_ENABLE_SQLITE_REPLICA=1 for best performance
6. Restart Space, all data will auto-persist to MySQL

PostgreSQL Configuration Example:

CCLOAD_POSTGRES=postgres://user:password@host:5432/ccload?sslmode=require

URL and libpq keyword DSNs are supported. Do not set CCLOAD_MYSQL and CCLOAD_POSTGRES at the same time.

Dockerfile Example (Pure External Database):

FROM ghcr.io/caidaoli/ccload:latest
ENV TZ=Asia/Shanghai
ENV PORT=7860
# Configure CCLOAD_MYSQL or CCLOAD_POSTGRES in Secrets; SQLITE_PATH is not required
EXPOSE 7860

Option 3: Local Storage Only (Not Recommended)

• ⚠️ Data Loss: /tmp clears on Space restart, channel config lost
• ⚠️ Manual Recovery: Must re-import via Web interface or CSV
• Use case: Temporary testing only

Update Deployment

With pre-built images, updates are simple:

Image Refresh:

• When new version image (ghcr.io/caidaoli/ccload:latest) is released
• Click "Factory rebuild" in Space settings to pull latest image
• Or wait for Hugging Face auto-restart (typically after 48 hours)

Manual Trigger Update:

# Add empty commit to trigger rebuild
git commit --allow-empty -m "Trigger rebuild to pull latest image"
git push

Version Pinning (Optional): To lock specific version, modify Dockerfile:

FROM ghcr.io/caidaoli/ccload:v2.44.1  # Specify version
ENV TZ=Asia/Shanghai
ENV PORT=7860
ENV SQLITE_PATH=/tmp/ccload.db
EXPOSE 7860

Basic Configuration

Choose SQLite, MySQL, or PostgreSQL based on the deployment shape. MySQL and PostgreSQL are mutually exclusive.

SQLite Mode (Default):

# Set environment variables
export CCLOAD_PASS=your_admin_password
export PORT=8080
export SQLITE_PATH=./data/ccload.db

# Or use .env file
echo "CCLOAD_PASS=your_admin_password" > .env
echo "PORT=8080" >> .env
echo "SQLITE_PATH=./data/ccload.db" >> .env

# Start service
./ccload

MySQL Mode:

# 1. Create MySQL database
mysql -u root -p -e "CREATE DATABASE ccload CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;"

# 2. Set environment variables
export CCLOAD_PASS=your_admin_password
export CCLOAD_MYSQL="user:password@tcp(localhost:3306)/ccload?charset=utf8mb4"
export PORT=8080

# Or use .env file
echo "CCLOAD_PASS=your_admin_password" > .env
echo "CCLOAD_MYSQL=user:password@tcp(localhost:3306)/ccload?charset=utf8mb4" >> .env
echo "PORT=8080" >> .env

# 3. Start service (auto-creates tables)
./ccload

PostgreSQL Mode:

# 1. Create the database and user in PostgreSQL

# 2. Set environment variables
export CCLOAD_PASS=your_admin_password
export CCLOAD_POSTGRES="postgres://user:password@localhost:5432/ccload?sslmode=disable"
export PORT=8080

# 3. Start service (auto-creates and migrates tables)
./ccload

Docker + MySQL:

# Option 1: docker-compose (Recommended)
cat > docker-compose.mysql.yml << 'EOF'
version: '3.8'
services:
  mysql:
    image: mysql:8.0
    environment:
      MYSQL_ROOT_PASSWORD: rootpass
      MYSQL_DATABASE: ccload
      MYSQL_USER: ccload
      MYSQL_PASSWORD: ccloadpass
    volumes:
      - mysql_data:/var/lib/mysql
    ports:
      - "3306:3306"
    healthcheck:
      test: ["CMD", "mysqladmin", "ping", "-h", "localhost"]
      interval: 10s
      timeout: 5s
      retries: 5

  ccload:
    image: ghcr.io/caidaoli/ccload:latest
    environment:
      CCLOAD_PASS: your_admin_password
      CCLOAD_MYSQL: "ccload:ccloadpass@tcp(mysql:3306)/ccload?charset=utf8mb4"
      PORT: 8080
    ports:
      - "8080:8080"
    depends_on:
      mysql:
        condition: service_healthy

volumes:
  mysql_data:
EOF

docker-compose -f docker-compose.mysql.yml up -d

# Option 2: Direct run (requires existing MySQL service)
docker run -d --name ccload \
  -p 8080:8080 \
  -e CCLOAD_PASS=your_admin_password \
  -e CCLOAD_MYSQL="user:pass@tcp(mysql_host:3306)/ccload?charset=utf8mb4" \
  ghcr.io/caidaoli/ccload:latest

Docker + PostgreSQL (requires an existing PostgreSQL service):

docker run -d --name ccload \
  -p 8080:8080 \
  -e CCLOAD_PASS=your_admin_password \
  -e CCLOAD_POSTGRES="postgres://user:pass@postgres_host:5432/ccload?sslmode=require" \
  ghcr.io/caidaoli/ccload:latest

After service starts, access:

• Admin Interface: http://localhost:8080/web/
• API Proxy: POST http://localhost:8080/v1/messages
• API Token Management: http://localhost:8080/web/tokens.html - Configure API access tokens via Web interface

📖 Usage Guide

API Proxy

Claude API Proxy (Requires Auth):

First, configure API access token in Web admin interface http://localhost:8080/web/tokens.html, then use that token to access API:

curl -X POST http://localhost:8080/v1/messages \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer your-api-token" \
  -H "x-api-key: your-claude-api-key" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
    "model": "claude-sonnet-4-6",
    "max_tokens": 1024,
    "messages": [
      {
        "role": "user",
        "content": "Hello, Claude!"
      }
    ]
  }'

OpenAI Compatible API Proxy (Chat Completions):

curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer your-api-token" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {
        "role": "user",
        "content": "Hello!"
      }
    ]
  }'

Codex Alpha Search (Native Passthrough Only):

POST /v1/alpha/search accepts the native Codex search payload. The model field is optional. This endpoint is forwarded only to channels whose resolved upstream protocol is Codex; it is not handled by local cross-protocol transforms.

curl -X POST http://localhost:8080/v1/alpha/search \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer your-api-token" \
  -d '{
    "query": "golang channels"
  }'

For a regular channel base URL, ccLoad appends /v1/alpha/search. If the channel uses the trailing # exact-URL marker, the configured URL must already point to this endpoint, for example https://upstream.example.com/v1/alpha/search#. Responses-only fields prompt_cache_key and prompt_cache_retention are removed before forwarding.

Local Token Counting

Quickly estimate request token consumption (no upstream API call needed):

curl -X POST http://localhost:8080/v1/messages/count_tokens \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet-4-6",
    "messages": [
      {"role": "user", "content": "Hello, how are you?"}
    ],
    "system": "You are a helpful assistant."
  }'

# Response example
# {
#   "input_tokens": 28
# }

Features:

• ✅ Compliant with Anthropic official API spec
• ✅ Local computation, <5ms response, no API quota consumption
• ✅ 93%+ accuracy (compared to official API)
• ✅ Supports system prompts, tool definitions, large-scale tool scenarios
• ✅ Requires auth token (configure at /web/tokens.html)

Channel Management

Manage channels via Web interface /web/channels.html or API:

# Add channel (supports multiple URLs, comma-separated)
curl -X POST http://localhost:8080/admin/channels \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Claude-API",
    "api_key": "sk-ant-api03-xxx",
    "url": "https://api.anthropic.com,https://api2.anthropic.com",
    "priority": 10,
    "rpm_limit": 0,
    "max_concurrency": 0,
    "models": ["claude-sonnet-4-6", "claude-opus-4-6"],
    "enabled": true
  }'

Multi-URL Note: The url field supports comma-separated multiple URLs. The system uses latency-weighted random selection for optimal URL choice, with automatic cooldown for failed URLs, enabling URL-level load balancing and failover within a single channel.

RPM Limit Note: rpm_limit is a per-channel request cap over a rolling 60-second window; 0 means unlimited. Proxy forwarding, manual tests, single-URL tests, and scheduled checks all count toward the cap. Multi-URL failover counts each actual upstream HTTP request. The counter is in-memory: restart clears it, and multiple instances count independently.

Concurrency Limit Note: max_concurrency is a per-channel cap on simultaneous in-flight upstream requests; 0 means unlimited. A slot is acquired before the upstream request starts and released when the response body is closed, so streaming requests hold the slot until the stream ends. Over-limit channels are skipped without cooldown. The counter is in-memory and per instance.

Custom Request Rules (Advanced)

The "Advanced" button in the channel editor opens a secondary modal that lets you rewrite the HTTP headers and JSON request body forwarded upstream at channel granularity. Typical use cases include User-Agent override, forcing API version headers, or tweaking fields like thinking / max_tokens. Rules apply in configured order and take effect for all subsequent requests on that channel as soon as they are saved.

Action matrix:

JSON path syntax:

• Dotted path + numeric array index: thinking.budget_tokens, messages.0.role, generation_config.temperature
• Values accept any JSON literal: number 0.7, boolean true, string "claude-opus-4-6", object {"type":"adaptive"}, array ["a","b"]

Safety constraints (hard-enforced server-side even if the frontend is bypassed):

• Auth header blacklist: any rule targeting Authorization, x-api-key, or x-goog-api-key (case-insensitive) is silently ignored and logged via slog.Warn
• CRLF injection guard: header names/values must not contain \r\n
• Non-JSON body passthrough: requests without application/json content type, empty bodies, or bodies that fail to deserialize are forwarded untouched without blocking
• Capacity caps: ≤ 32 header rules and ≤ 32 body rules per channel, each value ≤ 8 KB; violations return HTTP 400

Typical example:

{
  "custom_request_rules": {
    "headers": [
      { "action": "override", "name": "User-Agent", "value": "claude-cli/1.0 (custom)" },
      { "action": "remove",   "name": "Anthropic-Beta", "value": "context-1m-2025-08-07" },
      { "action": "append",   "name": "Accept", "value": "application/json" }
    ],
    "body": [
      { "action": "override", "path": "thinking", "value": {"type":"adaptive"} },
      { "action": "override", "path": "max_tokens", "value": 4096 },
      { "action": "remove",   "path": "stop_sequences" }
    ]
  }
}

Interaction with built-in logic: Custom rules run after the anyrouter anthropic-beta injection and anyrouter adaptive-thinking fallback, so they can override or remove those fields. Generated Anthropic requests use thinking.type=adaptive plus output_config.effort for thinking depth; anyrouter /v1/messages additionally fills missing thinking and normalizes legacy thinking.type=enabled. Authentication headers remain unmodifiable at all times.

Batch Data Management

Supports CSV format for channel config import/export:

Export Config:

# Web interface: Visit /web/channels.html, click "Export CSV" button
# API call:
curl -H "Authorization: Bearer your_token" \
  http://localhost:8080/admin/channels/export > channels.csv

Import Config:

# Web interface: Visit /web/channels.html, click "Import CSV" button
# API call:
curl -X POST -H "Authorization: Bearer your_token" \
  -F "[email protected]" \
  http://localhost:8080/admin/channels/import

CSV Format Example:

name,api_key,url,priority,models,enabled
Claude-API-1,sk-ant-xxx,https://api.anthropic.com,10,"[\"claude-sonnet-4-6\"]",true
Claude-API-2,sk-ant-yyy,https://api.anthropic.com,5,"[\"claude-opus-4-6\"]",true

Features:

• Auto column name mapping (Chinese/English)
• Smart data validation with error messages
• Incremental import and overwrite update
• UTF-8 encoding, Excel compatible

📊 Monitoring Metrics

Check out the awesome admin dashboard 👇

Real-time Monitoring Dashboard: Claude Code, Codex, OpenAI, and Gemini platform metrics at a glance

Core Features:

• 📈 24-hour Trend Charts - Request volumes clearly visualized with peaks and valleys

• 🔴 Real-time Error Logs - Instantly detect which channel has issues

• 📊 Channel Call Statistics - See which channels are performing well with data-backed insights

• 💬 Model Testing Workbench - Test by channel, by model, or in a chat-style model testing workflow:

  • Upload or paste images in chat mode to verify multimodal requests directly
  • Toggle reasoning level, built-in search, and streaming to inspect transformed upstream behavior
  • Export conversations as Markdown or HTML for review, incident notes, or regression records

• ⚡ Performance Metrics - Latency, success rates, and bottleneck detection

• 💰 Token Usage Stats - Know exactly where your budget goes:

  • Custom time range selector for flexible analysis
  • Per API token ID classification for multi-tenant billing
  • Supports Gemini/OpenAI cache token visualization

• 🎛️ Log Column Customization - Click the gear icon to show/hide columns, settings auto-saved to browser

UI Highlights:

• 🎨 Modern gradient purple theme for comfortable viewing
• 📱 Responsive design works great on mobile and desktop
• ⚡ Real-time data refresh without manual page reload
• 📊 Multi-dimensional stat cards show key metrics on one screen
  • Cached query optimization
  • Gemini/OpenAI Cache Token (Cache Read) display

🔧 Tech Stack

Core Dependencies

Architecture Features

Modular Architecture:

• Proxy Module Split (SRP):
  • proxy_handler.go: HTTP entry, concurrency control, route selection
  • proxy_forward.go: Core forwarding logic, request building, response handling
  • proxy_error.go: Error handling, cooldown decisions, retry logic
  • proxy_util.go: Constants, type definitions, utility functions
  • proxy_stream.go: Streaming responses, first byte detection
  • proxy_gemini.go: Gemini API special handling
  • proxy_sse_parser.go: SSE parser (defensive handling, Gemini/OpenAI cache token parsing)
  • proxy_debug.go: Upstream request/response debug capture (with sensitive header masking)
• Admin Module Split (SRP):
  • admin_channels.go: Channel CRUD
  • admin_stats.go: Stats analysis API
  • admin_cooldown.go: Cooldown management API
  • admin_csv.go: CSV import/export
  • admin_types.go: Admin API type definitions
  • admin_auth_tokens.go: API access token CRUD (with token stats, cost limits, model/channel restrictions, concurrency limits)
  • admin_settings.go: System settings management
  • admin_models.go: Model list management
  • admin_testing.go: Channel testing (with protocol transform testing)
  • admin_debug_log.go: Debug log API (sensitive header masking + base64 binary encoding)
  • channel_check_scheduler.go: Scheduled channel check scheduler
  • detection_log.go: Detection result to LogEntry builder
• Protocol Transform System (2026-07 core refresh):
  • protocol/types.go: Four protocol definitions (Anthropic/OpenAI/Gemini/Codex)
  • protocol/registry.go: Contract boundary for request, streaming response, and non-stream response transforms; same-protocol traffic bypasses conversion
  • protocol/builtin/register.go: Registers all 12 directed cross-protocol pairs
  • protocol/builtin/cliproxy_adapter.go: ccLoad-owned request validation, JSON/SSE normalization, and stream framing
  • protocol/cliproxy/: In-tree snapshot of the pure CLIProxyAPI conversion core; provenance and synchronization rules live in UPSTREAM.md
  • Upstream refresh workflow: invoke $sync-cliproxy-core in Codex or /sync-cliproxy-core in Claude Code; both resolve to the same repository Skill under .agents/skills/
  • Requests that cannot be represented in the selected upstream protocol return 400 Bad Request; they do not trigger channel failover or cooldown
  • Two modes: upstream (default, handled natively by upstream) / local (local translation)
  • Channel config: ProtocolTransformMode + ProtocolTransforms
  • Codex /v1/alpha/search is native passthrough only and never enters local protocol translation
• Cooldown Manager (DRY):
  • cooldown/manager.go: Unified cooldown decision engine
  • Eliminates duplicate code, unified cooldown logic
  • Distinguishes network vs HTTP error classification
  • Uses separate Key/Model/Channel actions; ActionRetryModel does not retry another Key or URL in the same channel
  • Persists structured model_cooldown responses, upstream HTTP 5xx failures, key-level 429 rate limits, and model-unavailable 404 errors by (channel_id, actual upstream model); other models on that channel stay eligible
  • Automatically promotes to channel cooldown only when all configured models or all enabled keys are cooling
• Multi-URL Selector (URLSelector):
  • url_selector.go: Smart URL selection within a single channel
  • Explore-first: Unvisited URLs get priority to collect latency data
  • Weighted random: Weight = 1/EWMA latency, lower latency = higher selection probability
  • Independent cooldown: Failed URLs cool down independently without affecting other URLs
  • BaseURL tracking: Active requests, logs, and UI carry upstream URL throughout
• Storage Layer Refactor (2025-12 optimization, eliminated 467 lines of duplicate code):
  • storage/schema/: Unified schema definition (supports SQLite/MySQL/PostgreSQL differences)
  • storage/sql/: Common SQL implementation layer shared by SQLite, MySQL, and PostgreSQL
  • storage/factory.go: Factory pattern auto-selects database
  • Composite index optimization, stats query performance improved
• OpenAI service_tier Pricing (2026-03 new):
  • util.OpenAIServiceTierMultiplier(): Returns multiplier for priority/flex/default tiers
  • LogEntry.ServiceTier: Persisted to database, log cost column shows tier annotation
  • Supports GPT-5.4, GPT-5.4-pro, and other latest model pricing
• Responses image_generation Tool Billing (2026-05 new):
  • Parses Responses API tool_usage.image_gen and the image_generation tool model
  • Bills gpt-image-2 by text input, image input, and image output tokens
  • Streaming/non-streaming proxy paths and channel tests share the same usage parser to keep cost accounting consistent
• Tiered Pricing:
  • GPT-5.4: Input price auto-steps down after token threshold
  • Qwen-Plus: Lower price tier kicks in after threshold
  • Gemini long-context: Price doubles above threshold
  • Cache discounts: Claude/Opus independent multipliers, OpenAI cache hit 50% discount

Multi-level Cache System:

• Channel config cache (60s TTL)
• Round-robin pointer cache (in-memory)
• Channel/Key cooldown state inline (channels / api_keys); model cooldown state in channel_model_cooldowns
• Error classification cache (1000 capacity)

Async Processing Architecture:

• Log system (1000 buffer + single worker, guarantees FIFO order)
• Token/log cleanup (background goroutine, periodic maintenance)

Unified Response System:

• StandardResponse[T] generic struct (DRY)
• ResponseHelper utility class with 9 shortcut methods
• Auto-extracts app-level error codes, unified JSON format

Connection Pool Optimization:

• SQLite: 10 connections for memory mode / 5 for file mode, 5-minute lifetime
• HTTP client: 100 max connections, 30s timeout, keepalive optimization
• TLS: Session cache (1024 capacity), reduces handshake latency

🔧 Configuration

Environment Variables