Mosaic Bridge

AI agents that drive the Unity Editor.

Mosaic Bridge connects MCP-compliant AI clients — Claude Code, Claude Desktop, Cursor, Gemini CLI, and any other Model Context Protocol client — to a running Unity Editor. It exposes ~290 tools covering GameObject and scene operations, procedural generation, physics simulation, pathfinding, rendering, animation, and more.

Free and open source under Apache 2.0. No telemetry. Your project never leaves your machine.

See it work

"Build me a Flappy Bird clone in Unity"

48 minutes later: a working game. No code written by hand, no manual editor clicks — Claude drove every Unity action through Mosaic Bridge tools.

▶ Watch the full build on YouTube

  You:     "Create a 3D tic-tac-toe board with warm wood styling and
            glowing cyan X pieces"
  Claude:  calls mosaic_gameobject_create, mosaic_material_create (uses
            KB for oak albedo), mosaic_physics_add-rigidbody, …
  Unity:   board appears in your scene, ready to play

What makes it different

• Research-paper implementations. ~90 of the tools are original implementations of published CS, graphics, and AI papers — Bridson's Poisson Disk (2007), Müller's SPH fluid (2003), Stam's Stable Fluids (1999), Orkin's GOAP (F.E.A.R. 2003), Lorensen-Cline Marching Cubes (1987), Reynolds' Boids (1987), Harabor-Grastien Jump Point Search (2011), Garland-Heckbert quadric decimation (1997). Not wrappers over Unity APIs — actual algorithms, callable by AI.

• Curated knowledge base. Tools that return physical quantities (density, friction, IOR, color temperature) draw from a bundled reference library — NIST physics constants, Unity's PhysicallyBased material data. Responses include citations. No hallucinated numbers.

• Multi-Unity-version. Works on Unity 2022 LTS and Unity 6. Each Editor runs in its own per-project runtime directory, so multiple Editors on different projects can run concurrently without fighting for ports or files.

• Privacy-first. Loopback-only HTTP listener bound to 127.0.0.1, HMAC-SHA256 authenticated, telemetry off by default. Ephemeral ports, per-project discovery files (mode 0600 on Unix).

• Vendor-neutral. Apache 2.0 with explicit patent grant. Not a funnel for a paid product. Your tools, your keys, your data.

Requirements

• Unity 2022 LTS or newer (Unity 6 fully supported)
• Node.js 18+ (for the MCP server)
• macOS, Windows, or Linux

Quick install

One command. It walks you through the rest:

npx @mosaicxr-ai/create-bridge

The installer:

1. Asks for your Unity project path (create one in Unity Hub first if you haven't — any empty Unity 2022 LTS+ or Unity 6 project works)
2. Asks which MCP client(s) to configure — Claude Code, Claude Desktop, Cursor, Gemini CLI, or OpenAI Codex (any combination)
3. Adds com.mosaic.bridge to the project's Packages/manifest.json
4. Writes the MCP server entry into each selected client's config

Then open the Unity project, wait for compile, restart your MCP client, and start prompting.

Non-interactive (CI / scripted)

npx @mosaicxr-ai/create-bridge \
  --project-path /path/to/UnityProject \
  --clients claude-code,cursor \
  --yes

See npx @mosaicxr-ai/create-bridge --help for all flags.

Scene-building intelligence + cross-LLM agents

The installer writes instruction files and specialist agents into your Unity project for every supported AI client:

All instruction files enforce:

• Interview before building — when you give a vague prompt like "make me a desert scene", the AI asks 4 targeted questions (scene type, geographic reference, scale, player perspective) before touching any tools.
• Spatial coherence — every placed object Y is resolved from terrain/sample-height before placement. No more objects buried underground or floating in air.
• Correct build order — terrain → textures → lighting → structures → vegetation → post-processing → camera.

Three specialist skill agents are installed following the bmad-method convention — written to both .claude/skills/ (Claude Code slash commands) and .agents/skills/ (universal, works with Cursor, Gemini, Codex, Windsurf, OpenCode):

The installer also copies four workflows into .claude/workflows/ (preflight, scene-plan, shader-guide, session-handoff).

Workflow prompts (any MCP client)

Beyond the instruction files, the MCP server exposes the same protocol rules as MCP prompts, so every client — not just ones with a CLAUDE.md — can pull them on demand: preflight, scene-interview, session-handoff, and shader-guide. In Claude Code they show up in the prompt picker; other clients list them via prompts/list.

To skip writing instruction files:

npx @mosaicxr-ai/create-bridge --skip-claude

To refresh after an update:

npx @mosaicxr-ai/create-bridge --force

Manual install (if you prefer)

Add to <UnityProject>/Packages/manifest.json:

{
  "dependencies": {
    "com.mosaic.bridge": "https://github.com/MosaicXR-AI/mosaic-bridge.git?path=/packages/com.mosaic.bridge"
  }
}

Then point your MCP client at:

{
  "mcpServers": {
    "mosaic-bridge": {
      "command": "npx",
      "args": ["-y", "@mosaicxr-ai/mcp-server", "--project-path", "/path/to/UnityProject"]
    }
  }
}

What's inside

~290 tools across 64 categories. A sample:

Plus: editor control (play mode, run-block, execute code, menu items), profiler, timeline, input system, constraints, LOD, assembly definitions, reflection, undo/redo, selection, tags & layers, settings, build, console, measurement, data visualization.

Full tool list: Packages/com.mosaic.bridge/Editor/Tools/ after install, or browse here.

Knowledge base

Tools that return physical values consult a bundled, versioned reference library:

• physics/constants.json — NIST CODATA fundamental constants
• rendering/pbr-materials.json — PhysicallyBased API material values (CC0)
• rendering/render-pipeline-compat.json — shader ↔ pipeline compatibility matrix (URP / HDRP / BuiltIn)
• rendering/shadergraph-nodes.json — 38 node aliases with input/output slot descriptions
• rendering/unity-api-quirks.json — documented API pitfalls with tested workarounds
• Planned: animation timing, audio attenuation, spatial metrics, color temperature, lighting presets

When a tool consults the KB, it attaches a citation to its response envelope so the AI knows which source a value came from. If a requested value isn't in the KB, the tool surfaces an explicit "no data available" warning rather than fabricating a number.

Supported MCP clients

Mosaic Bridge is protocol-native — it speaks Model Context Protocol 2024-11-05. The installer configures all of these in one pass:

Any MCP 2024-11-05 compliant client works with manual config. The auto-config list above is just the clients the installer knows paths/formats for today.

Multi-project: run several Unity Editors at once

Each Editor gets its own runtime directory under ~/Library/Application Support/Mosaic/Bridge/{projectHash}/ (macOS; analogous paths on Windows and Linux). State files never collide.

To have one MCP client talking to multiple Unity projects, register one MCP server per project:

{
  "mcpServers": {
    "mosaic-bridge-game": {
      "command": "node",
      "args": ["/path/to/mcp-server/dist/index.js", "--project-path", "/path/to/MyGame"]
    },
    "mosaic-bridge-factory": {
      "command": "node",
      "args": ["/path/to/mcp-server/dist/index.js", "--project-path", "/path/to/Factory"]
    }
  }
}

Each server connects to exactly the specified Editor. When you say "in my game project, create a cube", the client routes to the correct server by namespace.

Launched without --project-path, the server inspects the shared instance registry:

• Zero live Editors: clear error with setup guidance
• One live Editor: auto-selects it
• Two+ live Editors: structured error listing projects and the flags to disambiguate

MCP server flags

mosaic-mcp [options]
mosaic-mcp doctor [options]  Diagnose the bridge connection and exit

  --project-path <path>     Unity project root (recommended)
  --project-hash <hash>     16-hex-char project hash
  --runtime-dir <dir>       Explicit per-project runtime directory
  --discovery-file <file>   Direct path to bridge-discovery.json
  -h, --help                Show help
  -v, --version             Show version

Troubleshooting

Tools don't appear, or the client says the connection closed? Run the built-in doctor — it checks every link in the chain and tells you exactly which one is broken:

npx @mosaicxr-ai/mcp-server doctor --project-path /path/to/UnityProject

A healthy bridge reports all green:

  ✓ Discovery file: Found and signed (…/bridge-discovery.json), Unity 6000.3.10f1, port 8282.
  ✓ Live editor: 1 running Unity Editor (pid 56258, port 8282).
  ✓ Port reachable: 127.0.0.1:8282 accepted a connection.
  ✓ Health + HMAC: Bridge healthy — 265 tools, state Running.
  ✓ Clock skew: System clock within tolerance of the bridge host.

  Result: OK — the bridge is reachable.

Common failures and fixes:

With no --project-path and exactly one Editor open, doctor (and the server itself) auto-detects it — the simplest way to avoid path mistakes.

Architecture

                 Unity Editor (main thread)
                         │
                         │  [MosaicTool] attribute discovery via TypeCache
                         │
                ┌────────▼────────┐
                │  Mosaic Bridge  │   Unity C# package
                │   Core + Tools  │   In-process, main-thread dispatched
                └────────┬────────┘
                         │
                         │  HTTP on loopback (127.0.0.1, ephemeral port)
                         │  HMAC-SHA256 challenge-response authentication
                         │  Discovery file at {project-hash} runtime dir
                         │
                ┌────────▼────────┐
                │  MCP Server     │   Node.js / TypeScript
                │  stdio ↔ MCP    │
                └────────┬────────┘
                         │
                         │  Model Context Protocol over stdio
                         │
      ┌──────────────────┼──────────────────┬──────────────┐
      │                  │                  │              │
  Claude Code      Claude Desktop        Cursor        Gemini CLI

Key properties:

• In-process bridge — Unity APIs are main-thread-only, cross-process calls throw UnityException
• Main-thread dispatch via EditorApplication.update — guaranteed safe
• Survives Unity domain reloads via SessionState persistence
• TypeCache discovery — add a static method with [MosaicTool(...)], it's automatically callable
• Per-project isolation — multiple Editors never share mutable state files
• Telemetry off by default

Development

git clone https://github.com/MosaicXR-AI/mosaic-bridge.git
cd mosaic-bridge

# MCP server
cd packages/mcp-server
npm install
npm run build
npm test

# Installer CLI
cd ../create-bridge
npm install
npm test

# Unity package — install via file: reference in a test project's manifest.json
# See TESTING.md for the full test workflow (all three suites).

Monorepo layout:

packages/
├── com.mosaic.bridge/       Unity UPM package (Editor + Runtime + Tests)
│   ├── Editor/              ~290 tools + core infrastructure
│   ├── Runtime/             Runtime-compatible tool subset
│   ├── Tests/               NUnit + Unity Test Runner
│   └── Samples~/            Custom-tool authoring sample
└── mcp-server/              Node.js MCP server (TypeScript + Vitest)

Running Unity tests requires opting in via testables in the consuming project's manifest.json:

{
  "testables": ["com.mosaic.bridge"]
}

Roadmap

v1.0 beta (current)

• Core bridge, MCP server, ~290 tools across 64 categories
• Per-project runtime isolation
• Auto .mcp.json for Claude Code + auto-config for Claude Desktop, Cursor, Gemini CLI, and OpenAI Codex CLI via npx @mosaicxr-ai/create-bridge
• Windows cmd /c wrapper for stdio MCP launch (beta.2)
• Scene intelligence: scene/create-object decision tree (project search → Asset Store → procedural build), spatial coherence tools, build plans
• editor/run-block multi-statement C# execution with polling
• Knowledge base with Unity-version-aware guidance + rendering compat KB (beta.3)
• project/preflight — render pipeline + color property detection (beta.3)
• material/create-batch — bulk material creation (beta.3)
• Cross-LLM specialist agents via bmad-method SKILL.md format (beta.3): Zara, Ray, Max installed to .claude/skills/ + .agents/skills/
• Unity project asset resources in MCP (@Unity Prefabs, @Unity Materials, etc.) (beta.3)
• Workflow rules exposed as MCP prompts — preflight, scene-interview, session-handoff, shader-guide — so every client receives them (beta.7)
• mosaic-mcp doctor — one-command connection diagnostics (discovery file, live editor, port, HMAC handshake, clock skew) (beta.7)
• Apache 2.0 license with patent grant

v1.0 stable

• Unity Asset Store listing
• OpenUPM registry publication
• First docs site
• Runtime kb/query + kb/fetch server contract live
• kb/watch background drift detector for Unity-docs updates

v1.1+

• Migrate deprecated Unity API usage to modern equivalents
• Expand knowledge base (animation, audio, level design, lighting presets)
• Runtime (compiled build) tool support for more categories
• Performance profiler integration

See open issues for the live backlog, and Discussions for design conversations.

Contributing

Contributions welcome — pull requests, issues, test coverage, docs. Every commit must be signed off per the Developer Certificate of Origin:

git commit -s -m "your message"

See CONTRIBUTING.md for the full workflow, and CODE_OF_CONDUCT.md for community standards.

Security vulnerabilities: please disclose privately — see SECURITY.md.

License

Apache License 2.0. Includes an explicit patent grant.

Individual research-backed tools cite the papers they implement. Algorithm citations are embedded in the source code and returned in tool response envelopes where relevant.

Credits

Built and maintained by Mousa Soutari / MosaicXR.

Research-paper implementations respectfully credit their authors. Knowledge base values are sourced from public authoritative data (NIST, Unity PhysicallyBased, etc.) under their respective licenses.

If you build something interesting with Mosaic Bridge, open a Discussion — we'd love to see it.