Claude Code Local: On-Device AI for Apple Silicon

🤔 What Is This?

Your Mac has a powerful GPU built right into the chip. This project uses that GPU to run massive AI models — the same kind that power ChatGPT and Claude — entirely on your computer, and plugs them into Claude Code so the whole coding experience works offline.

🚫 No internet needed 💰 No monthly subscription 🔒 No one sees your code or data ✅ Full Claude Code experience — write code, edit files, manage projects, control your browser, or run a full hands-free voice session

         📱 You (Mac or Phone)
          │
     🤖 Claude Code           ← the AI coding tool you know
          │  HTTP localhost:4000
     ⚡ MLX Native Server      ← this repo (~1000 lines of Python)
          │
     🥊 Pick your fighter     ← Gemma 4 31B · Llama 3.3 70B · Qwen 3.5 122B
          │
     🖥️  Apple Silicon GPU    ← your M-series chip does all the work

The trick: Claude Code speaks the Anthropic API. Local model servers speak the OpenAI API. So everyone bolts a translation proxy in between — and the proxy is slow and fragile. This server speaks Anthropic natively. One process, zero translations:

🎯 That one change — eliminating the proxy — made it 7.5× faster.

🎬 Watch It Run — AirGap AI

A real NDA. Llama 3.3 70B. Wi-Fi physically OFF. lsof running live. Watch a 70-billion-parameter model audit a confidential legal document, on-device, with the receipts on screen.

More local-AI demos on the channel:

🥊 The Lineup — Pick Your Fighter

We started with one model. Now we ship a roster. Same MLX server, same Anthropic API — swap one env var and you swap the brain. Plus the ds4 engine for DeepSeek V4 Flash via its own native Metal runtime.

💡 Fun fact: Qwen wins raw speed because it's an MoE — only 10B of 122B params activate per token. DeepSeek V4 Flash is even bigger (284B) but only ~37B active per token, and it ships with on-disk KV cache so a 25k-token Claude Code system prompt prefills exactly once, ever.

🐳 DeepSeek V4 Flash via ds4

We tested it the day Antirez (the Redis guy) shipped ds4. Local DeepSeek beat cloud Claude on wall-clock time on the same MacBook, same prompt — watch the three-way.

⭐ Our Own MLX Abliterated Uploads

The models in this lineup aren't from generic mirrors — we package and upload our own abliterated MLX builds to HuggingFace so anyone running this repo can pull them with one command. Browse the full set at huggingface.co/divinetribe.

# Llama 3.3 70B — full-precision feel
MLX_MODEL=divinetribe/Llama-3.3-70B-Instruct-abliterated-8bit-mlx \
  bash scripts/start-mlx-server.sh

# Gemma 4 31B — fast daily driver
MLX_MODEL=divinetribe/gemma-4-31b-it-abliterated-4bit-mlx \
  bash scripts/start-mlx-server.sh

# Hermes 4 14B — sweet spot for 16/32 GB Macs
MLX_MODEL=divinetribe/Hermes-4-14B-abliterated-4bit-mlx \
  bash scripts/start-mlx-server.sh

Abliteration sources: huihui-ai (Llama, Gemma) and Babsie (Hermes). MLX conversion + quantization by us. See what abliteration means.

⚠️ Use it responsibly. "Abliterated" suppresses the model's built-in refusal direction so it doesn't refuse benign-but-edgy requests. It is not a general capability upgrade, and you remain bound by each upstream license (Llama 3.3, Gemma, Hermes/Qwen3).

🎮 The Modes

Four ways to run the lineup. Each one is a double-clickable launcher in launchers/.

💻 What You Need

Also need:

• 🐍 Python 3.12+ (for MLX)
• 🤖 Claude Code (npm install -g @anthropic-ai/claude-code)

🚀 Quick Start (3 Commands)

git clone https://github.com/nicedreamzapp/claude-code-local
cd claude-code-local
bash setup.sh

setup.sh auto-detects your RAM, picks a model from the lineup, downloads it, installs the MLX server, and creates a Claude Local.command launcher on your Desktop.

Then double-click Claude Local.command. You're coding locally.

🐛 If the launcher asks you to sign in to a Claude account: your claude CLI is too old. The launchers pass --bare to force local-only API-key auth; older CLIs don't support it. Fix: npm install -g @anthropic-ai/claude-code

🛠️ Note for contributors: setup.sh installs the server as a symlink at ~/.local/mlx-native-server/server.py pointing back at this repo's proxy/server.py. Edit the file in the repo, restart the server, done — one source of truth, no silent drift.

Or do it manually

# 1. Set up the MLX virtualenv
python3.12 -m venv ~/.local/mlx-server
~/.local/mlx-server/bin/pip install mlx-lm

# 2. Pick a fighter and download (one time, ~18-75 GB)
bash scripts/download-and-import.sh gemma   # or 'llama' or 'qwen'

# 3. Start the server
MLX_MODEL=divinetribe/gemma-4-31b-it-abliterated-4bit-mlx \
  bash scripts/start-mlx-server.sh

# 4. Launch Claude Code
ANTHROPIC_BASE_URL=http://localhost:4000 \
ANTHROPIC_API_KEY=sk-local \
claude --model claude-sonnet-4-6

🔧 How It Works

┌──────────────────────────────────────────────────┐
│              Your MacBook (M-series)             │
│                                                  │
│  📝 You type ──> 🤖 Claude Code                  │
│                      │                           │
│                      ▼                           │
│                 ⚡ MLX Server (port 4000)        │
│                      │                           │
│                      ▼                           │
│                 🥊 Local model ──> 🖥️  GPU        │
│                 (Gemma·Llama·Qwen)               │
│                      │                           │
│                      ▼                           │
│  📝 Answer <─── ✨ Clean response                │
│                                                  │
│         🔒 Nothing leaves this box. Ever.        │
└──────────────────────────────────────────────────┘

The server (proxy/server.py) is one file, ~1000 lines. It does six things:

1. 📦 Loads the model — Apple's MLX framework, native Metal GPU, unified memory. Handles Gemma's RotatingKVCache quirk automatically.
2. 🔌 Speaks Anthropic API — Claude Code thinks it's talking to Anthropic's cloud. It's not.
3. 🔧 Translates tool use — Three tool-call formats in and out: Gemma 4 native, Llama 3.3 raw JSON, and HuggingFace <tool_call> JSON (Qwen and others). All converted ↔ Anthropic tool_use blocks, with garbled-output recovery for small models.
4. 🧹 Cleans the output — A real-time ThinkingFilter strips <think> blocks token-by-token during generation, then clean_response handles stop markers and reasoning preamble.
5. ⚡ Reuses prompt caches across requests — Claude Code's system prompt doesn't get re-prefilled every turn. Huge speedup for short questions.
6. 🎯 Code mode — auto-detects Claude Code coding sessions, swaps the ~10K-token harness prompt for a slim ~150-token one, and strips verbose tool descriptions to name + parameter types. A 28× prompt reduction that cuts prefill from ~60 s to ~2 s on Gemma 4 31B.

🛤️ The Journey

We didn't start here. Three generations in one night:

🔒 Privacy + How the Data Flows

This is the part we're proudest of. Your code never leaves your Mac. Not for a model call. Not for telemetry. Not for "anonymous analytics". Not ever.

   ┌─────────────────────────────────────────────────────────────┐
   │                    🖥️  YOUR MACBOOK                          │
   │                                                             │
   │   📝 Your code ──> 🤖 Claude Code ──> ⚡ MLX Server          │
   │                     (localhost:4000)      │                 │
   │                                           ▼                 │
   │                    🧠 Local model ──> 🖥️  Apple GPU          │
   │                                                             │
   │             🚫 ZERO outbound network calls                  │
   │             🚫 ZERO telemetry                               │
   │             🚫 ZERO phone-home                              │
   └─────────────────────────────────────────────────────────────┘
                   │
                   ✗  ←  Nothing from our code crosses this line.
                   │
   ┌─────────────────────────────────────────────────────────────┐
   │                    ☁️  THE INTERNET                          │
   │                  (your code never goes here)                 │
   └─────────────────────────────────────────────────────────────┘

🔍 What We Audited (Every Component)

✅ Verified offline. Claude Code's own binary previously reached out to api.anthropic.com on startup for telemetry, statsig feature flags, marketplace auto-install, and the autoupdater. The launchers plug all four channels via documented Anthropic env vars (thanks @tadrianonet, PR #32):

CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1
DISABLE_AUTOUPDATER=1
CLAUDE_CODE_DISABLE_OFFICIAL_MARKETPLACE_AUTOINSTALL=1
CLAUDE_CODE_DISABLE_BACKGROUND_TASKS=1

Verify it yourself: run lsof -p $(pgrep -f claude) during a session — you'll see only localhost:4000. Run lsof -i -P while the server is up — nothing leaves your Mac.

⚠️ We removed LiteLLM after supply-chain attack concerns. Every dependency was re-audited from scratch. If a package had unexplained network calls, it didn't ship.

✈️ When To Use This

📊 Benchmarks

Three generations of optimization. Each one got faster.

⚡ Speed Comparison

🥊 Lineup Comparison

☁️ vs Cloud APIs

💡 Our local setup beats cloud Opus on raw speed (65 vs 40 tok/s) at $0/month. Qwen numbers measured on M5 Max 128 GB — full details in BENCHMARKS.md.

🔧 Tool-Call Reliability

Local models don't format tool calls perfectly. They want to call a tool but mix XML and JSON syntax — Claude Code sees no valid tool call, re-prompts, and the model garbles it the same way again. The result: infinite loops where the AI says "let me do that" but never does anything.

We fixed this with 4 changes to server.py:

🧪 Results: 98/98 tests passed across 7 consecutive runs. Zero failures. The multi-step scenario that used to trigger infinite loops — create 12 month folders, delete all but September, verify — now passes every time. Run it yourself:

python3 scripts/test_mlx_server.py

⚙️ Tuning

🔌 MCP Servers — Claude Code's plugin ecosystem, 100% local

The only way to run Claude Code's full MCP plugin ecosystem 100% local on Apple Silicon.

Most local-LLM proxies break MCP — they strip tool definitions, mangle tool_use blocks, or refuse to forward the streaming format Claude Code expects. This server passes tool definitions through to your local model and translates the responses back into Anthropic's format, across all three model families. From Claude Code's perspective it's talking to Anthropic. From your MCP server's perspective, nothing changed.

Wire servers up the normal Claude Code way:

# Filesystem — let the local model read/write a folder
claude mcp add filesystem -- npx -y @modelcontextprotocol/server-filesystem ~/projects

# GitHub — issues, PRs, code search
claude mcp add github --env GITHUB_TOKEN=$GITHUB_TOKEN -- npx -y @modelcontextprotocol/server-github

# Web search — for when the local model needs fresh info
claude mcp add brave-search --env BRAVE_API_KEY=$BRAVE_API_KEY -- npx -y @modelcontextprotocol/server-brave-search

The whole 200+ server MCP universe works the same against your local Gemma or Qwen — just running on your machine instead of someone else's.

🎤 Hands-Free Voice Mode

Talk to your Mac. It talks back in your own cloned voice. Nothing touches the internet in either direction — most "AI voice" demos use cloud STT and cloud TTS; this runs both sides of the loop fully on-device.

┌─────────────────────────────────────────────────────┐
│              YOUR MACBOOK (M-series)                │
│                                                     │
│  🎙️  Your voice                                     │
│      ▼                                              │
│  🎧 listen   ← Apple SFSpeechRecognizer, on-device  │
│      ▼         continuous, stability-based cutoff   │
│  ⌨️  inject  ← AppleScript → Terminal window        │
│      ▼                                              │
│  🤖 claude → ⚡ MLX Server → 🥊 Gemma 4 31B          │
│      ▼                                              │
│  🔊 speak    ← cloned-voice TTS (Pocket TTS/Piper)  │
│      ▼                                              │
│  👂 You hear it — and keep talking                  │
│                                                     │
│   🔒 Your voice never leaves this box. Ever.        │
└─────────────────────────────────────────────────────┘

• 🎙️ Speech-in — a compiled Swift binary wraps Apple's on-device SFSpeechRecognizer in a continuous listening loop. End of utterance = transcript stable for 2.5s — way more robust than silence heuristics against fans or music.
• 🔊 Speech-out — ~/.local/bin/speak wraps a cloned-voice TTS. Any TTS that takes text and plays audio slots in: macOS say, Piper, Pocket TTS.
• 🔁 Feedback-loop prevention — the listener auto-pauses during playback so the model never hears itself.
• 🛡️ Production hardening — 10-minute preventive recycle (dodges a known SFSpeech daemon wedge), queue-backlog detection. Runs unattended for hours.

The two halves:

• 🗣️ Speak-and-think (this repo): launchers/Narrative Gemma.command boots the MLX server with the narration persona (NarrativeGemma/CLAUDE.md) so Gemma narrates every tool call and result out loud.
• 🎧 Listen-and-inject (NarrateClaude, sibling repo): the Swift listener, dispatch pipeline, and one-click narrative-claude.sh launcher.

# 1. This repo — MLX server + Narrative launcher
git clone https://github.com/nicedreamzapp/claude-code-local.git && cd claude-code-local && bash setup.sh

# 2. Sibling repo — the listening pipeline
git clone https://github.com/nicedreamzapp/NarrateClaude.git ~/NarrateClaude
cd ~/NarrateClaude && chmod +x dictation/bin/* narrative-claude.sh
./dictation/bin/dictation setup

# 3. Launch the full hands-free loop
bash ~/NarrateClaude/narrative-claude.sh

🌐 Browser Agent

A standalone agent that controls your real Brave browser via Chrome DevTools Protocol — powered entirely by local AI. Lives in its own repo: nicedreamzapp/browser-agent. The Browser Agent.command launcher here starts the MLX server, opens Brave with remote debugging, and drops you into the agent.

     📝 Your task
      ▼
 🤖 agent.py              ← autonomous browser agent (separate repo)
      ▼
 ⚡ MLX Server             ← local AI decides what to do
      ▼
 🌐 Brave (CDP port 9222) ← clicks, types, navigates your real browser
      ▼
 📊 Context Meter          ← color-coded memory usage after each step

📱 Control From Your Phone

You don't have to be at your Mac to use this. Text a command, get back a full video.

📱 Your iPhone                    💻 Your Mac
     │                                │
     │── "find me an article  ──────>│── imessage-receive.sh reads it
     │    and send me a video"        │── local model plans the task
     │                                │── Brave browser finds the article
     │                                │── speak narrates in your voice
     │                                │── Studio Record captures it all
     │<── 🎥 video in iMessage ──────│── imessage-send-video.sh ships it
     │                                │
   🛋️  From your couch            🖥️  At your desk

Everything works — text, images, and video:

Full pipeline repo: nicedreamzapp/claude-screen-to-phone → Clone it, run setup.sh, fill in your phone number. Works with this local AI stack or Claude cloud. Docs in IMESSAGE_MEDIA_PIPELINE.md.

📁 What's In This Repo

📦 claude-code-local/
 ├── ⚡ proxy/
 │   └── server.py              ← MLX Native Anthropic Server with tool-call recovery (~1000 lines)
 ├── 🚀 launchers/
 │   ├── Claude Local.command    ← Default fighter — Claude Code + local model
 │   ├── Gemma 4 Code.command    ← 🟢 THE QUICK ONE
 │   ├── Llama 70B.command       ← 🟠 THE WISE ONE
 │   ├── Browser Agent.command   ← 🌐 Autonomous Brave browser control
 │   ├── Narrative Gemma.command ← 🎭 Auto-narration mode
 │   └── lib/claude-local-common.sh ← Shared: model-aware restart, cache resolver, health-wait
 ├── 🎭 NarrativeGemma/
 │   └── CLAUDE.md              ← Narration persona (sanitized, generic, opt-in)
 ├── 🛠️  scripts/
 │   ├── download-and-import.sh ← Download a fighter (`gemma` / `llama` / `qwen`)
 │   ├── persistent-download.sh ← Auto-retry downloader for big models
 │   ├── start-mlx-server.sh    ← Server start helper
 │   ├── test_mlx_server.py     ← Tool-call reliability test suite
 │   └── upload-mlx-quant.sh    ← Publish your own MLX-quantized uploads to HF
 ├── 📊 docs/
 │   └── BENCHMARKS.md          ← Detailed speed comparisons
 └── setup.sh                    ← One-command installer

🧩 The Local-First Stack

claude-code-local is the brain. It pairs with sibling repos — each stands alone, together they take Claude Code off the keyboard and off the screen:

🛣️ What's Next

We ship fast and in public. If any of these excite you, hit Watch to get the release ping.

• 🟡 Full Qwen 3.5 122B benchmark suite — reliability, tool-call pass rate, long-context behavior vs Gemma
• 🟡 Fully-local Whisper fallback — alternative to the Apple SFSpeechRecognizer path for older Macs and non-English voices
• 🟡 One-click DMG installer — no terminal needed
• 🟡 MLX_MODEL=<hf-url> — point at any HuggingFace repo and auto-register a new fighter
• 🟡 More fighters — open to PRs adding launchers for DeepSeek, Mistral, Phi, anything MLX-compatible

💡 Want something that's not on this list? Open an issue → Every serious request gets read and usually replied to within 24h.

🤝 Contributing

Ideas, bug reports, a new launcher for a model I don't run, a better code-mode prompt — open an issue or a PR, I read them all. Especially interested in: folks on older Apple Silicon (M1/M2, 16–36 GB) who know which models actually fit; anyone stress-testing the voice loop on different hardware or accents; TTS recipes beyond Pocket TTS (Piper, MLX-TTS, Kyutai Moshi); and edge cases I'll never hit on an M5 Max with 128 GB.

🙏 Credits

Built on the shoulders of giants:

Tested on Apple M5 Max with 128 GB unified memory.

Built by Matt Macosko in Arcata, CA — part of Nice Dreamz LLC. More open-source at nicedreamzwholesale.com/software · demos at youtube.com/@nicedreamzapps.