coach

Agents have potential, but they need reinforcement to fulfill it. Coach provides that locally, from an agent's own past executions — developing the skills it lacks, heading off errors it has made before, turning repeated actions into code (muscle memory), and dropping to cheaper, dumber models where they're good enough (auto-pilot). The result: agents that are faster, cheaper, and more accurate.

🎥 Demo — coming soon.

Until the video lands, the same moment is written up in text: docs/case-study.md walks coach pointed at its author's own ~148 Claude Code sessions, ranking its mistakes by preventable cost — a worked example that runs entirely on the shipped query surface.

Key Observations

• Much of what agents call an LLM to do could be a script. A large share of agentic "workflows" are LLM inference where a deterministic program would do — editing a package.json, formatting a file, applying a known transform. The script version is deterministic, fast, and cheap; the inference version is none of those. Once you know which steps occur and how often, you can tell which are worth converting into code rather than paying for a one-off inference every time (is it worth the time?).
• Agent observability is aimed at engineers, but the data is too big for engineers. The platforms that exist surface traces for a human to read, yet the volume of session data is a glacier — a person can barely touch its edge, let alone mine the insights buried in it. There is, however, exactly the right technology for reading that much data: agents.
• The agent's control flow should be derived, not hand-drawn. Early agents were built as explicit graphs; that felt too rigid, so the field moved to reactive (ReAct-style) loops. The better path looks like neither hand-authored extreme: derive the graph from what the agent actually did and let it adapt continuously and automatically. Coach's execution graph — recovered in hindsight from real traces — is the first step toward that.
• Coding agents are the beachhead, and the Claude ecosystem is the most mature one. The most widely used agents today are coding agents, and within them Claude Code is the most adopted and mature. That is where the richest traces — and the most leverage — currently live.
  • Most developers don't own their session data. Claude Code already writes detailed logs of every session, and for almost everyone they sit unused. That data is a standing asset; coach turns it into a queryable model instead of leaving it on the floor.

What works today

It works on OpenTelemetry (OTEL) traces by design, which keeps coach harness-agnostic. The shipped surface is three things: a pure, staged pipeline, a React Flow visualization, and a read-only MCP query server.

• Pipeline (@coach/pipeline) — turns trace/log files into a normalized, id-keyed execution graph, a model that maps 1:1 to a relational DB. Pure and file-system-free; runs in Node and the browser alike.
• Visualization (@coach/app) — a React Flow renderer for a pre-computed execution graph.
• MCP query server (@coach/mcp) — exposes that graph as a read-only, queryable relational surface so an analyst agent drives its own analyses over your sessions.

For the full picture — package layout, pipeline stages, data flow, the MCP query surface, and deployment — see ARCHITECTURE.md. It is the living source of truth; this README only covers getting started.

Quick start

Requires pnpm (do not use npm/yarn) and Node ≥ 20.11.

pnpm install
pnpm check                       # typecheck + lint + format + test + knip (same as CI)

Produce an execution graph and inspect the per-stage output:

pnpm e2e <dir | repo-name>       # writes out/<name>/01-classified.json … 07-resolved-graph.json + graph.db

The argument is either a directory of traces (a path relative to cwd, e.g. a fixture under packages/pipeline/fixtures/) or a repo name like coach, which loads that repo's Claude Code logs across the main checkout and every git worktree — the same convenience the MCP's load_dataset exposes.

View a graph in the browser:

pnpm --filter @coach/app dev     # landing page at http://localhost:5173

Load a 07-resolved-graph.json (or any pre-computed graph) via the "Load pipeline output" button, or boot directly from a URL with ?data=<url>; add ?focus=<nodeId> to reveal and center a node.

Use it from Claude Code (MCP)

Coach ships an MCP server (@coach/mcp) that exposes the analyzed execution graph as a read-only, queryable surface — so the agent can drive its own analyses over your sessions. Claude Code is the only supported agent for now.

1. Register the server. Run from anywhere — use the absolute path to this repo so it resolves regardless of where Claude Code launches the server:

# preload a dataset (a directory of OTEL trace/log JSON, or native .jsonl sessions)
claude mcp add coach -- node --experimental-strip-types \
  /ABSOLUTE/PATH/TO/coach/packages/mcp/bin/mcp.ts /ABSOLUTE/PATH/TO/traces

# …or omit the directory and load data at runtime via the load_dataset tool
claude mcp add coach -- node --experimental-strip-types \
  /ABSOLUTE/PATH/TO/coach/packages/mcp/bin/mcp.ts

The server speaks MCP over stdio. The optional trailing directory is preloaded so the dataset is queryable immediately; without it, ask the agent to call load_dataset with a path first. During development you can also run it directly with pnpm mcp [dataset-dir] from the repo root.

2. Verify and use it. In a Claude Code session, run /mcp to confirm "coach" is connected, then ask in plain language — e.g. "load the traces in ./out and find the most expensive interactions". The server exposes:

To remove it later: claude mcp remove coach.

A standalone DuckDB snapshot for ad-hoc inspection in the duckdb CLI is also available via pnpm build-db <traces-dir> [out.db] (the MCP itself re-derives from source rather than loading it).

For a worked analysis built from these tools — failed tool calls ranked by recovery cost, with the SQL — see docs/case-study.md.

Development

The full command reference, quality gates, and module conventions live in CLAUDE.md.

Where this is going

Everything in this section is roadmap — not yet shipped. It states design intent, not current behavior. Today coach targets the engineer: you load a dataset and query it. The two threads below are about whom the findings ultimately serve.

• Closing the feedback loop back to the agent. The north star is for the agent to act on its own findings, not just for an engineer to read them — Claude Code loading its own sessions and correcting its own expensive, hallucinated, or wasteful steps. How that loop is "closed" is still undecided, so stage one deliberately targets the engineer until we learn which problems are actually solvable.
• The cross-session per-agent user model (the second pillar). Because coach holds complete sessions and many of them, it aims to infer user intent in hindsight and roll it up across sessions into a per-agent user model — what users want, how they phrase it, what they leave unsaid, what needs a clarifying question — a personalization signal population-level RLHF cannot produce. This is a separate output with a separate consumer (the agent, for personalization); it does not replace the engineer-facing optimization work. The conceptual model — and its honest caveats — lives in docs/agent-model.md.