FerrumDeck

FerrumDeck is a deterministic Rust enforcement plane for AI agents — it blocks the tool call in-process, it doesn't just chart it after the fact. Deny-by-default tool policy, per-run budget enforcement, runtime (Airlock) inspection, approval gates, and an append-only audit trail — all in the request path, returning an allow/deny/approve decision before the agent acts.

Enforce, don't just observe. LangSmith, Phoenix, Galileo, and Fiddler watch your agent and tell you afterward what it did. FerrumDeck sits in the call path and enforces — it returns allowed=false and the tool never fires. Observability is a dashboard you read after the incident; enforcement is the gate that prevents it. And because the gate sits in the trace, every decision it makes is itself a queryable OTel GenAI span (ferrumdeck.decision = allow|deny|approval|kill) — you enforce and observe in one pass, not two tools.

See the blind spot for yourself. docs/benchmarks/enforce-vs-observe.md runs one AgentDojo-style injection trace two ways over the same governance profile: a record-only stack that emits a span after the unsafe send_email already ran, versus the in-path gate that emits ferrumdeck.decision=deny on the same span and the call never fires. Deterministic, offline, no LLM — spans captured with an in-memory exporter so the output is real telemetry, not a mock. Reproduce with make bench-enforce-vs-observe.

"But won't in-path enforcement slow my agent down?" No — the decision is sub-millisecond. Measured CPU cost of the governance decision itself (Apple M4, --release, decision path only — excludes DB / queue / LLM):

An LLM step costs hundreds of ms to seconds; ~1 µs of in-path governance is ~6 orders of magnitude smaller than the call it gates. Reproduce with make bench-enforcement; methodology + full table (incl. deny / RCE-blocked cases) in docs/benchmarks/enforcement-latency.md. This is the added decision cost, not end-to-end latency — no "fastest"/"first" claim.

▶ Run the 5-minute reproducible demo → — one command boots the local stack and, against the real gateway API, you watch a budget-breach auto-kill and a denied tool call happen in-process. It's self-verifying: each guarantee is asserted with jq and the script exits non-zero on failure, so you get a hard pass/fail, not a screenshot to trust.

Status: early / alpha, built primarily by one maintainer. The governance core — per-agent deny-by-default tool allowlists, per-run/per-agent budget enforcement, DB-backed tenant isolation, and Airlock RASP at the gateway tool-policy check — is implemented and tested. Several advertised layers are still being wired end-to-end. See Project Status & Limitations for an honest map of what enforces today vs. what's on the roadmap before you rely on it.

Install from crates.io

The enforcement engine is published — you can depend on it, not just clone it. One dependency via the umbrella crate:

cargo add ferrumdeck

use ferrumdeck::{PolicyEngine, ToolAllowlist};

let engine = PolicyEngine::default();
let allowlist = ToolAllowlist { allowed_tools: vec!["read_file".into()], approval_required: vec![], denied_tools: vec!["delete_repo".into()] };
assert!(engine.evaluate_tool_call_with(&allowlist, "read_file").is_allowed()); // allowed
assert!(engine.evaluate_tool_call_with(&allowlist, "delete_repo").is_denied()); // denied
assert!(engine.evaluate_tool_call_with(&allowlist, "unknown").is_denied());     // deny-by-default

Prefer the engine directly (no umbrella)? cargo add ferrumdeck-policy — the import path is still use fd_policy::…. Primitives only: ferrumdeck-core (use fd_core::…). All three are Apache-2.0. (Published as ferrumdeck* because the bare fd-core name is taken on crates.io by an unrelated crate; the Rust import paths are unchanged.)

What this proves

FerrumDeck is the control plane, not the agent — the production layer that makes an autonomous agent safe to run: it decides which tools a run may call, kills runs that blow their budget, gates risky actions on a human, and records every decision in an immutable trail. It's built as a credibility artifact for an AgentOps / AI-infrastructure audience.

▶ Run the 5-minute reproducible demo → — one command boots the local stack and verifies, against the real gateway API, the four guarantees below:

• Deny-by-default tool policy — a run may only call tools on its per-agent allowlist; everything else is denied. (POST /v1/runs/{id}/check-tool)
• Budget auto-kill — every run carries a hard token / cost / tool-call / wall-time budget; a breach kills the run and appends a budget.exceeded event. (fd_policy::budget → RunStatus::BudgetKilled)
• Coherence-divergence caught mid-run — when an agent states a blocking fact ("tests failing", "permission denied") and then advances as if it were untrue, the live monitor catches it on the run stream and applies the reversibility ladder (R1–R3). (fd_policy::airlock::coherence)
• Immutable audit trail — every policy, budget, and approval decision is appended to audit_events; the repository exposes no UPDATE/DELETE.
• OTel GenAI spans — and every enforce decision is one — every LLM/tool step emits OpenTelemetry GenAI-semconv spans to Jaeger, and so does every enforcement decision: the tool-policy check writes a gen_ai.tool.call span carrying ferrumdeck.decision = allow|deny|approval|kill, ferrumdeck.reason, ferrumdeck.rung (R1–R3), and ferrumdeck.budget_remaining, so the allow/deny you enforce is the span you query. Naming follows the GenAI-semconv stability opt-in (OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental → execute_tool + gen_ai.operation.name); Rust gateway and Python worker write one schema. (fd_otel::decision · fd_runtime.tracing)

The demo is self-verifying — it asserts each property with jq and exits non-zero on failure, so it works as a smoke test, not a screenshot. For an honest map of what enforces today vs. what's still being wired, see Project Status & Limitations.

Reproduced: indirect-prompt-injection defense-path coverage

Against a vendored AgentDojo-style (arXiv:2406.13352) indirect-injection corpus (25 cases: 17 attacks — off-allowlist tools, RCE payloads, data-exfil destinations — and 8 benign controls), FerrumDeck's deny-by-default tool allowlist + Airlock RASP block:

One-command repro (deterministic, offline, no LLM required):

uv run python -m fd_evals injection-defense --suite injection_defense   # or: make eval-injection-defense

Honest framing: this measures defense-path coverage on a fixed governance profile — the fraction of injected malicious tool calls the policy/RASP layer blocks — not model robustness, and not a general "injection-proof" claim. The corpus is pinned to the real Rust fd_policy RASP by cargo test -p fd-policy --test injection_defense (which runs the actual AirlockInspector over every case), and the fd-evals mirror must agree with it. Small vendored corpus ⇒ a wide CI; the number moves as the corpus grows. No "first"/"best" claim.

Second axis: Agent Security Bench (ASB) + an EU AI Act Art. 50 transparency rule

The injection_defense axis above is AgentDojo (indirect injection). The asb axis adds a second, complementary set: attack classes from Agent Security Bench (arXiv:2410.02644) that AgentDojo does not cover — the Plan-of-Thought (PoT) backdoor, memory poisoning, and direct prompt injection — run through the same governance path plus the R1–R3 reversibility ladder. The point of difference: a backdoored plan that reaches an allowlisted-but-irreversible action (a deploy, an apply_migration) is stopped not by the allowlist but by the R3 rung (irreversible → require_approval, so it never auto-executes). That is graduated enforcement catching an attack a static allowlist would wave through.

Of the 13 ASB attacks, 4 are stopped specifically by the R3 reversibility rung (the PoT-backdoor class), the rest by deny-by-default, the anti-RCE matcher, and the exfiltration shield.

The EU AI Act Article 50 rule (fd_policy::transparency_art50) is a transparency enforcement rule on the same R1–R3 ladder: a governed generative response must carry a human-readable AI disclosure and a machine-readable synthetic-content marker; in enforce mode a response missing either is denied before release (R3), in shadow it is logged (R1).

One-command repro (deterministic, offline, seeded, no LLM):

uv run python -m fd_evals asb --suite asb --seed 0   # or: make eval-asb

Honest framing: the ASB number measures whether the malicious action is stopped from auto-executing on a fixed governance profile — not that FerrumDeck detects the backdoor trigger or the poisoned memory semantically, and not model robustness (no LLM). The Art. 50 rule is a structural check (presence of a disclosure phrase + a machine-readable marker), not a truthfulness or standard-conformance (C2PA/SynthID) judgement. Both are pinned to the real Rust enforcement by cargo test -p fd-policy --test asb_defense — which runs the actual ToolAllowlist + AirlockInspector + reversibility ladder and the real transparency_art50 rule over every case, and asserts the R3 rung is the decisive layer for the PoT class — and the fd_evals.asb mirror must agree. Small vendored corpus ⇒ wide CIs. No "first"/"best" claim.

Live-fire: a drifting run is caught and R-tiered (deterministic, no stack)

examples/demo/coherence-drift.py feeds a deliberately drifting trajectory through the same detection core the live gateway runs on each step, and is self-verifying (exits non-zero if the drift is not caught). Real captured output:

▸ Drifting run — states a blocking fact, then advances as if it were untrue
  ✗ DIVERGENCE [test_failure] confidence=1.00
      stated:       '2 tests failed: assertion error in auth flow'
      then did:     'git_commit: commit: fix auth flow'
      response:     R3 (require_approval)   shadow → record + surface   enforce → GATE run → WaitingApproval
      sse coherence.divergence.detected {category:test_failure, response_rung:R3, gated(enforce):True}
  ✗ DIVERGENCE [permission_denied] confidence=1.00
      stated:       'deploy blocked: permission denied on prod'
      then did:     'deploy: shipping to production, reporting success'
      response:     R3 (require_approval)   shadow → record + surface   enforce → GATE run → WaitingApproval

▸ Coherent control — resolves the fact before advancing (must NOT fire)
  ✓ no divergence — the resolved run is clean

COHERENCE OK ✓  2 divergences fired (both R3), control clean. In enforce mode each R3 gates the run for review.

Against the live gateway the same divergence writes an audit_events row with violation_type=coherence_divergence, emits the coherence.divergence.detected SSE event, and surfaces on the run's Coherence card. It runs shadow by default (records + surfaces, never blocks); set FERRUMDECK_COHERENCE_MODE=enforce to gate an R3 divergence (run → WaitingApproval). This is the enforce-not-observe wedge: FerrumDeck acts on the drift, it doesn't just chart it.

Mapping to the DeepMind AI Control Roadmap (D1–D4 / R1–R3)

Google DeepMind's AI Control Roadmap (2026-06-18, blog) frames agent security on two axes: Detection tiers D1–D4 (set by the model's detection-evasion capability — how hard a misaligned agent is to catch) and Prevention & Response tiers R1–R3 (set by the model's attack-execution capability — how strong the response must be). FerrumDeck already names this ladder in fd_policy::reversibility (anchor deepmind-ai-control-roadmap-r1-r3). To be precise and avoid overclaiming: D1–D4 is an adversary-capability rating, not something a defense tool "is" — FerrumDeck maps onto the R-tier response axis and the roadmap's operational metrics, not onto a self-assigned detection tier.

Response tier → FerrumDeck mechanism (R3 = synchronous block-before-execute; R1 = asynchronous review-after-the-fact):

Detection → the roadmap's live metrics. DeepMind measures a control system by coverage (fraction of agent traffic monitored), recall (fraction of misaligned behaviors caught), and time-to-response:

• Coverage — the CoherenceMonitor observes the run's trajectory event stream (each submitted step), so coverage is the fraction of the run stream it sees. (Honest limit: today it is fed at the gateway step-submission choke point, not the in-loop agentic executor — see Project Status & Limitations.)
• Recall — the divergence pattern is the Strained-Coherence signal (arxiv:2606.07889); the shared cross-plane golden fixture pins detection so recall doesn't silently regress.
• Time-to-response — CoherenceMonitor::observe_event fires mid-run, the instant a divergence completes, before the run finishes — not only in a post-hoc autopsy. That is what moves the same signal from R1 (async) to R3 (synchronous) when enforce is on.

Honest scope. R3 enforcement of a coherence divergence is opt-in (FERRUMDECK_COHERENCE_MODE=enforce) and shadow-by-default — the heuristic is a lexical matcher with a non-zero false-positive rate, so gating is a deliberate choice for high-consequence agents (the roadmap's own point: R1 async review is insufficient for critical-infrastructure agents where a harmful action can't be undone after the fact). The mapping above is machine-checked: examples/demo/coherence-drift.py prints and asserts the R3 → "DeepMind R3: synchronous block-before-execute" label and that the enforce path gates it, exiting non-zero if that ever stops holding. Several advertised layers remain in-progress — see Project Status & Limitations before relying on any tier.

Table of Contents

• What This Proves
  • Mapping to the DeepMind AI Control Roadmap (D1–D4 / R1–R3)
• Overview
• Project Status & Limitations
• Key Features
• Architecture
• Quick Start
• Project Structure
• Components
  • Control Plane (Rust)
  • Data Plane (Python)
  • Dashboard (Next.js)
• API Reference
• Configuration
• Security Model
• Observability
• Evaluation Framework
• Development
• Deployment
• License

Overview

FerrumDeck solves the critical challenge of running AI agents safely in production. While LLMs are probabilistic and unpredictable, production systems require deterministic governance, audit trails, and budget controls.

The Problem

• AI agents can make costly mistakes (token spend, wrong tool calls)
• Prompt injection attacks can bypass safety measures
• No visibility into what agents are doing in production
• Difficult to reproduce and debug agent failures
• Compliance requirements demand audit trails

The Solution

FerrumDeck provides a dual-plane architecture:

Project Status & Limitations

FerrumDeck is an early-stage / alpha project, built primarily by a single maintainer. It is a real, working control plane — but it is not yet production-hardened. This is an honest map of what enforces today vs. what is scaffolded or on the roadmap, so you can evaluate it without surprises.

Implemented and enforced (covered by the Rust test suite):

• Deny-by-default tool policy, per agent. The gateway evaluates every tool call against the run's agent allowlist (allowed / approval-required / denied tiers) — not a process-global default.
• Budget enforcement, per run / per agent. The auto-kill and the cost forecast evaluate against the run's effective budget (per-run config.budget override → agent-version caps → engine default).
• Tenant isolation. Project-scoped access is gated by a DB-backed project → workspace → tenant ownership check; unknown project or tenant mismatch is denied.
• Airlock RASP at the gateway tool-policy check (POST /v1/runs/{id}/check-tool): the anti-RCE pattern matcher, the financial/velocity circuit breaker, and the data-exfiltration + credential-DLP shield run here, in shadow or enforce mode.
• Append-only audit trail for policy, budget, approval, routing, and promotion decisions (the repository exposes no UPDATE/DELETE).
• Coherence-divergence monitor, wired live at the gateway run stream. As each step is submitted, the run's trajectory is fed to the CoherenceMonitor; a stated-blocking-fact → contradicting-closure-action divergence surfaces mid-run through the same airlock.violation_detected audit path, is persisted on the run row (coherence_divergence_flagged), and emitted on the completion span. A reliability signal — it never blocks a tool or kills a run.

Scaffolded / not yet wired end-to-end — do not rely on these yet:

• Airlock on the agentic execution path. The Python worker's agentic LLM-loop executor does not yet call back to the control-plane check-tool endpoint, so Airlock and approval gates are enforced on the explicit StepType.TOOL path but not inside an in-loop agentic run. Wiring this is the top roadmap item; until then, run agentic workloads only in trusted contexts.
• Schema-drift and behavioral-drift Airlock layers are implemented and unit-tested but are not activated in the running gateway (they need tool_version_id / agent_id plumbed into the inspection context).
• Trace→signal loop (HarnessX). The harness-suggestion governance endpoints (/v1/harness-suggestions*) and the training-signal export (POST /v1/runs/{id}/training-signal, redacted server-side via the audit redaction path) are implemented, unit-tested, and wired into the dashboard. But the evals dashboard data is still BFF-stubbed (/api/v1/evals/* returns empty until a gateway eval backend lands), so the full eval→gateway→dashboard round-trip is demonstrable only with a live stack and a non-stub eval feed. Approving a suggestion records the decision; it never auto-applies a policy/allowlist/budget change.
• Audit tamper-evidence. The log is append-only at the application layer, but there is no cryptographic hash-chain or DB-level write-once enforcement yet — so it is not tamper-evident against a privileged database actor. A hash-chain is on the roadmap; please don't represent the trail as immutable/tamper-proof for compliance until it ships.
• Multi-tenant SaaS hardening. Tenant isolation is enforced, but there is no dashboard auth/session layer, no SSO/RBAC, and no API-key self-service — treat the dashboard + gateway as a trusted-operator deployment for now.

Testing caveat. The unit/lint suites (cargo test --workspace, clippy, ruff, jest) pass and gate CI. The tests/security, tests/chaos, and tests/e2e suites require a live stack (make dev-up) and currently assert liveness more than behaviour — do not read them as proof that a given attack is blocked. Hardening them is in progress.

Found a gap not listed here? Please open an issue — accurate status is a feature.

Key Features

Governance

• Deny-by-Default Tools: Only explicitly allowed tools can be called
• Approval Gates: High-risk actions require human approval before execution
• Budget Enforcement: Automatic run termination when limits exceeded (tokens, cost, time)
• Predictive Budget Forecast: Deterministic linear + EWMA projection of end-of-run cost after every step, surfacing a budget_breach_projected flag on the run API + SSE event (run.forecast.updated) before the auto-kill fires. See docs/runbooks/budget-forecast.md.
• Policy Engine: Configurable rules for tool access and risk management
• Airlock RASP: Five runtime self-protection layers on every tool call — anti-RCE pattern matcher, financial circuit breaker, data-exfiltration shield, schema-drift guard, behavioral-drift monitor. Shadow or enforce modes.
• Explicit Conflict Resolution + Decision Traces: When multiple policies match a tool call, a named precedence function (Deny > RequiresApproval > BudgetCap > Allow) picks the winner deterministically, and every decision carries an audit-grade trace of matched verdicts and overrides surfaced on the run API + policy.decision.explained SSE event. See docs/runbooks/policy-conflict-resolution.md.
• Routing-Decision Audit (multi-agent coordination): Every time the orchestrator binds a subtask to a concrete agent / role / model, a RoutingDecision record (candidates considered, chosen binding, reason code, SHA-256 content hash) is written through the existing immutable audit trail and surfaced on GET /v1/runs/{id}/routing plus the routing.decision.recorded SSE event. fd-evals replays compare the content hash to detect coordination drift. Anchor: AgensFlow (arXiv:2605.27466). See docs/runbooks/routing-decision-audit.md.
• Champion-Challenger Promotion Gate: A registered challenger version cannot replace the live champion until it clears a deterministic gate — configurable metric thresholds (inclusive floors) plus a required human approval. Deny-by-default: the challenger stays in shadow until explicitly promoted. The decision + metric evidence (SHA-256 content hash for tamper-evidence) flow through the same PolicyDecision channel every gate uses and are written to the immutable audit trail. Exposed on POST /v1/promotions/evaluate (write scope) + GET /v1/promotions/{agent_id}, surfaced on the agent dashboard (champion vs challenger + gate status). See docs/runbooks/champion-challenger-promotion.md.
• Reversibility-Aware Graduated Response (R1–R3 ladder): A Reversibility { reversible, costly, irreversible } dimension on the tool registry — orthogonal to the existing risk tiers — drives a graduated response at the gateway tool-policy check, modelled on the DeepMind AI Control Roadmap R1–R3 ladder: reversible → R1 allow-and-log (monitor, no gate); costly → R2 allow-under-budget while the run's cost budget has headroom, escalating to R3 when exhausted; irreversible → R3 require-approval (the existing human-in-the-loop gate). Deny-by-default: an unregistered tool defaults to irreversible. The chosen rung is folded into the allowlist decision more-restrictive-wins (it can only add friction, never loosen a Deny), emitted on the OTel span (ferrumdeck.policy.response_level) + the immutable audit log, returned from POST /v1/runs/{id}/check-tool, and surfaced on the run console as an R1/R2/R3 badge (read from the polled run endpoint; the realtime policy.response.recorded SSE shape is defined, with gateway→BFF push deferred). See docs/runbooks/graduated-response-levels.md.
• Eval-Driven Harness Suggestions (trace→delta, HarnessX): fd-evals turns the aggregate signal across an eval run's trace into a proposed harness/policy delta — e.g. "run cost exceeded the cap on 7/10 runs → propose a tighter per-call cap" — and POSTs it to the control plane. The HarnessSuggestion is content-hashed and written to the immutable audit trail (same store as the promotion gate, no parallel channel), exposed on POST /v1/harness-suggestions + GET /v1/harness-suggestions/agent/{agent_id} + POST /v1/harness-suggestions/{id}/resolve, and surfaced on the eval-run dashboard with a review/approve panel. Human-in-the-loop, deny-by-default: approving records the decision in the audit trail and never auto-applies a change to a live policy, allowlist, or budget — applying remains a separate, explicit step.
• Delegation-Aware Budget Leases: The stateless budget gate compares an accumulated usage snapshot against a cap, which lets a parent task that delegates to N children collectively spend up to N × the cap — every child checking the same cap believes it owns the whole budget (the Token-Budgets delegation-fanout class). A BudgetLease closes that gap: all leases in one delegation tree share a single atomic remaining-budget pool, a child is handed a sub-lease carved from (not copied alongside) the parent's authority, and every spend decrements the one shared pool — so total spend across parent + children can never exceed the root cap, even under concurrent fan-out. The lease is move-only (!Copy, !Clone): a lease moved into a delegated child is a compile error if the parent reuses it, runtime-rejected otherwise. Anchor: Token Budgets (arXiv:2606.04056).

Observability

• OpenTelemetry Integration: Full distributed tracing with GenAI semantic conventions
• Cross-MCP trace correlation (MCP SEP-414): When a caller propagates W3C trace context in the tool-call request _meta (traceparent/tracestate/baggage), ferrumdeck parents its enforcement decision span on that context — so the decision joins your trace end-to-end (host → client SDK → MCP server → ferrumdeck decision → downstream) and the trace-id lands on the persisted decision record. Malformed traceparent is rejected (never propagated); off unless the OTel semconv opt-in is set. Targets the 2026-07-28 MCP revision (a Release Candidate) and implements the SEP-414 conventions — not a conformance claim. See docs/mcp-trace-conformance.md.
• Cost Tracking: Real-time token counting and cost calculation per run
• Jaeger UI: Visual trace exploration and debugging
• Audit Trail: Immutable logging of every action for compliance
• Tool-call firing rate: Derived OTel signal (ferrumdeck.metrics.tool_call_firing_rate) tracking the share of reasoning steps that invoked at least one tool, per run + per agent over a sliding window. Surfaced on the agent overview tab with a configurable low-firing-rate threshold (default 40%) that flags model regressions or broken tool registries before they propagate. See docs/runbooks/tool-call-firing-rate.md.
• Debt-vs-tax cost decomposition (§2605.27320): Per-call span_role ∈ {primary, retry, judge, guardrail, escalation, revalidation, monitor} classification on every LLM/tool call, with two derived rollups per task/run — agent.cost.token (primary calls = debt) and agent.cost.tax (everything else). Dashboard panel ranks tasks by tax / (token + tax) descending so retry / escalation storms are visible at a glance. See docs/runbooks/cost-decomposition.md.
• Claim grounding rate — grounding rate per VeriGraph (arXiv:2606.16603): A per-run reliability metric (ferrumdeck.reliability.claim_grounding_rate, 0.0–1.0) — the fraction of the final agent output's claims that are reachable from a raw-data / tool-output source node via the run's evidence graph, per VeriGraph's claim-level grounding definition. This is a lineage to the claim-level auditability literature, not a ferrumdeck-original metric. Computed at run completion (Rust fd_otel::claim_grounding, mirrored by Python fd_evals.claim_grounding for the eval plane, with a shared golden fixture pinning cross-plane agreement), persisted on the run row next to cost/tokens, emitted on the run span, and rendered as a stat card on the run console. Honest scope: the "reachable evidence path" is operationalized as a deterministic lexical-overlap reachability proxy (sentence-split claims; a claim is grounded when enough of its significant tokens are covered by a source node) — pure and CI-stable, not an LLM judge or semantic-entailment model. It is a reliability signal only: a project may set an optional min_claim_grounding_rate in its settings to flag (never block or kill) a run below it — off by default, preserving the deny-by-default posture for tool permissions, not reliability scoring. See docs/runbooks/claim-grounding-rate.md.

Reproducibility

• Versioned Registry: Agents, tools, and prompts are version-controlled
• Step-Level Replay: Debug specific steps with exact inputs
• Deterministic IDs: ULID-based identifiers for time-ordered, collision-resistant tracking

Quality

• Evaluation Framework: Deterministic test suites for agent workflows
• Regression Gating: CI blocks merges if agent quality degrades
• Baseline Comparisons: Track performance across versions
• Per-harness eval dimension (Harness-Bench): fd-evals reports at the (model × harness_config) level — same model under different harness configs can produce different scores. Each run records its tools_available, permission_tier, state_recovery, and tracing config alongside the existing baseline, the dashboard groups results by (model × harness) with a side-by-side Recharts bar chart, and DeltaReport exposes a per-dimension diff (added/removed tools, tier change, recovery change). See docs/runbooks/harness-config.md.
• Training-signal export (trace→signal, HarnessX): closes the eval loop the other way — projects a run's trace into a JSONL of (state, action, observation, outcome_score) tuples for downstream training/eval. Built server-side at POST /v1/runs/{id}/training-signal, where every state/observation is run through the existing audit redaction path (fd_audit::redaction) so PII/secrets are stripped before they ever leave the control plane; outcome_score is trace-intrinsic (step status) with an optional eval-supplied run_score override. The dashboard exposes a per-suite/per-run "Download training signal" action.

Architecture

┌─────────────────────────────────────────────────────────────────────────┐
│                              Clients                                      │
│              (Dashboard / CLI / SDK / CI Pipelines)                      │
└─────────────────────────────────────────────────────────────────────────┘
        │                           │                           │
        ▼                           ▼                           ▼
┌─────────────────┐    ┌──────────────────────────────────────────────────┐
│    DASHBOARD    │    │                CONTROL PLANE (Rust)               │
│   (Next.js)     │    │                                                   │
│                 │    │  ┌───────────┐  ┌──────────┐  ┌──────────────┐   │
│ • Runs Monitor  │◀──▶│  │  Gateway  │  │  Policy  │  │   Registry   │   │
│ • Approvals     │    │  │  (Axum)   │  │  Engine  │  │  (Versioned) │   │
│ • Analytics     │    │  │           │  │          │  │              │   │
│ • Audit Trail   │    │  │ • REST    │  │ • Budget │  │ • Agents     │   │
│ • Evals UI      │    │  │ • SSE     │  │ • Rules  │  │ • Tools      │   │
│                 │    │  │ • Auth    │  │ • Gates  │  │ • Versions   │   │
└─────────────────┘    │  └───────────┘  └──────────┘  └──────────────┘   │
   :3001/:8000         │                                                   │
                       │  ┌───────────┐  ┌──────────┐  ┌──────────────┐   │
                       │  │   Audit   │  │   DAG    │  │    OTEL      │   │
                       │  │    Log    │  │Scheduler │  │    Setup     │   │
                       │  └───────────┘  └──────────┘  └──────────────┘   │
                       └──────────────────────────────────────────────────┘
                                              │
                          ┌───────────────────┼───────────────────┐
                          ▼                   ▼                   ▼
                   ┌───────────────┐   ┌───────────────┐   ┌───────────┐
                   │   PostgreSQL  │   │     Redis     │   │   Jaeger  │
                   │   (pgvector)  │   │    Streams    │   │    UI     │
                   │               │   │               │   │           │
                   │ • runs/steps  │   │ • Job Queue   │   │ • Traces  │
                   │ • agents/tools│   │ • Pub/Sub     │   │ • GenAI   │
                   │ • audit_events│   │               │   │   Spans   │
                   └───────────────┘   └───────┬───────┘   └───────────┘
                        :5433                  │                :16686
                                               ▼
              ┌───────────────────────────────────────────────────────────┐
              │                      DATA PLANE (Python)                    │
              │                                                             │
              │  ┌──────────────┐  ┌──────────────┐  ┌──────────────────┐  │
              │  │    Worker    │  │     LLM      │  │    MCP Router    │  │
              │  │              │  │   Executor   │  │                  │  │
              │  │ • Poll Queue │  │              │  │ • GitHub MCP     │  │
              │  │ • Execute    │  │ • Claude     │  │ • Filesystem MCP │  │
              │  │ • Report     │  │ • GPT-4      │  │ • Custom Tools   │  │
              │  │ • Retry      │  │ • litellm    │  │ • Policy Checks  │  │
              │  └──────────────┘  └──────────────┘  └──────────────────┘  │
              └───────────────────────────────────────────────────────────┘

Data Flow

1. Client creates a run via POST /v1/runs
2. Gateway authenticates, validates, creates run in PostgreSQL
3. Gateway enqueues first step to Redis Stream
4. Worker polls Redis, fetches step details from Gateway
5. Worker executes step (LLM call, tool call, etc.) with tracing
6. Worker reports result back to Gateway
7. Gateway updates state, checks budget, enqueues next step
8. Repeat until run completes or fails

Service Ports

Receipts schema

The control plane's append-only audit log is documented as a stable receipts substrate compatible with Foundation Protocol (Mila + MetaGPT). See docs/receipts-schema.md for the canonical AuditEvent shape, the FP event-substrate mapping (metering / receipt / settlement / policy / provenance / audit), the wrap-don't-replace stance on downstream consumers, and the per-call p95 budget. Drift is gated by the audit_record_schema_drift integration test in rust/crates/fd-audit/tests/.

Quick Start

Just want to see it work? Run the one-command reproducible demo (./examples/demo/run-demo.sh) — it boots the stack and self-verifies deny-by-default policy, the approval gate, the immutable audit trail, and OTel spans in Jaeger.

Prerequisites

• Rust 1.80+ (rustup.rs)
• Python 3.12+
• Docker & Docker Compose
• uv ([docs.astral.sh/uv](https://docs.astral.sh/uv