FerrumDeck

FerrumDeck is a deterministic Rust enforcement plane for AI agents — it blocks the tool call in-process, it doesn't just chart it after the fact. Deny-by-default tool policy, per-run budget enforcement, runtime (Airlock) inspection, approval gates, and an append-only audit trail — all in the request path, returning an allow/deny/approve decision before the agent acts.

Enforce, don't just observe. LangSmith, Phoenix, Galileo, and Fiddler watch your agent and tell you afterward what it did. FerrumDeck sits in the call path and enforces — it returns allowed=false and the tool never fires. Observability is a dashboard you read after the incident; enforcement is the gate that prevents it. And because the gate sits in the trace, every decision it makes is itself a queryable OTel GenAI span (ferrumdeck.decision = allow|deny|approval|kill) — you enforce and observe in one pass, not two tools.

See the blind spot for yourself. docs/benchmarks/enforce-vs-observe.md runs one AgentDojo-style injection trace two ways over the same governance profile: a record-only stack that emits a span after the unsafe send_email already ran, versus the in-path gate that emits ferrumdeck.decision=deny on the same span and the call never fires. Deterministic, offline, no LLM — spans captured with an in-memory exporter so the output is real telemetry, not a mock. Reproduce with make bench-enforce-vs-observe.

"But won't in-path enforcement slow my agent down?" No — the decision is sub-millisecond. Measured CPU cost of the governance decision itself (Apple M4, --release, decision path only — excludes DB / queue / LLM):

An LLM step costs hundreds of ms to seconds; ~1 µs of in-path governance is ~6 orders of magnitude smaller than the call it gates. Reproduce with make bench-enforcement (or make reproduce-readme-figures to also compare against the numbers above and fail on drift); methodology + full table (incl. deny / RCE-blocked cases) in docs/benchmarks/enforcement-latency.md. This is the added decision cost, not end-to-end latency — no "fastest"/"first" claim.

Governed vs ungoverned (reproducible) — measured on a recognized public workload. The governance layer, run two ways (ungoverned = a record-only stack with no decision point; governed = the deny-by-default allowlist + Airlock RASP + spend gate deciding before execution), on citable, deterministic, offline workloads — every number regenerable and pinned to the real Rust fd_policy engine by a cargo test:

• Indirect prompt injection (AgentDojo-style, arXiv:2406.13352, 17 attack / 8 benign): attack success rate 100% → 0% (17/17 blocked; Wilson CI [81.6%, 100%]) with 100% benign-task utility retained (8/8) — every attack stopped, zero false-positives on this corpus.
• Spend-overrun (fixed safe-PR trajectory, 4 injected unsafe actions): 4/4 blocked vs 0/4 ungoverned, and the governed run costs 54% less (85¢ vs 184¢) because stopping the RCE / exfil / denied-tool / runaway loop saves more than the ~1 µs/decision + audit overhead.
• Payment-governance (an agent overspends an AP2 mandate / x402 call): 3/3 unsafe mandates blocked, $150.95 → $0.40 — each blocked on a distinct control (bad Ed25519 signature, over-ceiling, out-of-scope merchant), every verdict audit-logged (ferrumdeck.decision span + W3C traceparent).

Check every figure on this page yourself in one command: make reproduce-readme-figures re-measures all of it from a clean clone — the latency table, the block/benign rates with their Wilson intervals, and the spend figures — and exits non-zero on drift. Deterministic and offline: no services, no API keys, no money moved. Rates are compared exactly, because those benchmarks are seeded and LLM-free; latencies are compared within a stated band against the stated machine (Apple M4, --release), because a nanosecond figure that fails on different silicon is a broken gate rather than a strict one. The nightly runs it and opens a tracking issue on drift. Add --skip-latency (or make reproduce-readme-figures-fast) to skip the ~2-minute criterion sweep.

make reproduce-spend-gate remains the narrower check for just the two spend figures, driving the AP2 gate and the x402 example end to end.

Full table, workloads, reproduce commands + honest caveats: fd-evals/GOVERNED_BENCHMARK_RESULTS.md. Each control mapped to the five risk categories in the CISA/NSA (Five Eyes) Careful Adoption of Agentic AI Services (May 2026) guidance — and the transparency control tied to EU AI Act Article 50, enforceable 2026-08-02 — in fd-evals/CONTROLS_CROSSWALK.md. Reproduce: make eval-injection-defense · make bench-governed; method + numbers also in docs/BENCHMARK.md.

x402 spend gate — budget enforcement for autonomous payments. The x402 protocol (Coinbase-contributed, now stewarded by the x402 Foundation under the Linux Foundation, 2026-07-14; Cloudflare's Monetization Gateway began charging agents per access over it 2026-07-01) lets an agent pay for a paywalled resource inline — the server answers with HTTP 402 Payment Required and a stablecoin quote, and the agent pays and retries. FerrumDeck extends the same per-agent budget that caps token spend to that new category: it prices the 402 quote in cents (a first-class cost event on the same ledger as inference), checks it against the remaining budget before the payment is authorized, and hard-stops (deny + one alert) if paying would breach the ceiling. It moves no money — a simulate → gate → record demo only. This is the same posture platform vendors are converging on (Databricks Unity AI Gateway hard spend caps over any model/agent/MCP service; FinOps FOCUS 1.4 extending cost accounting to AI token/agent economics), done deny-by-default in-path. Run it: cargo run -p ferrumdeck --example x402_spend_gate — examples/x402-spend-gate (fd_policy::x402).

Payment-rail coverage: x402 + AP2. The same pre-call spend gate now covers a second rail — Google's AP2 (Agent Payments Protocol), where a payment is pre-authorized by a signed Mandate chain rather than an inline HTTP 402 quote. Before an autonomous payment is authorized, FerrumDeck verifies the Ed25519 signature chain (a user-signed Intent Mandate + a Cart Mandate cryptographically bound to it), checks the cart is within the intent's authorized scope (merchant/category + the user's own max), and checks the cart total against the same per-task Budget::has_cost_headroom ceiling the x402 gate uses. It is deny-by-default: a missing/invalid signature, an unknown key, a cart total over the ceiling, an amount over the intent max, or a merchant outside scope all hard-stop the payment — verifying real signatures, not trusting a flag. The authorized payment folds into the same cost_cents ledger as x402 + tokens, and emits the same governance evidence (W3C-trace-context decision span + an audit record). Verification is pinned by cargo test -p fd-policy --test ap2_gate; the governed-vs-ungoverned AP2 numbers ship as a row in the benchmark below (fd_policy::ap2).

▶ Run the 5-minute reproducible demo → — one command boots the local stack and, against the real gateway API, you watch a budget-breach auto-kill and a denied tool call happen in-process. It's self-verifying: each guarantee is asserted with jq and the script exits non-zero on failure, so you get a hard pass/fail, not a screenshot to trust.

Status: early / alpha, built primarily by one maintainer. The governance core — per-agent deny-by-default tool allowlists, per-run/per-agent budget enforcement, DB-backed tenant isolation, and Airlock RASP at the gateway tool-policy check — is implemented and tested. Several advertised layers are still being wired end-to-end. See Project Status & Limitations for an honest map of what enforces today vs. what's on the roadmap before you rely on it.

Install from crates.io

The enforcement engine is published — you can depend on it, not just clone it. One dependency via the umbrella crate:

Current version: v0.8.8. cargo add ferrumdeck pulls the latest published release. The --features audit variant below has resolved since 0.8.4 — the release that first published ferrumdeck-audit; on 0.8.0–0.8.1 that command errored, because the crate was unpublished and the name unclaimed. (This version line is asserted against the workspace version by a test, so it can't silently go stale.)

cargo add ferrumdeck

use ferrumdeck::{PolicyEngine, ToolAllowlist};

let engine = PolicyEngine::default();
let allowlist = ToolAllowlist { allowed_tools: vec!["read_file".into()], approval_required: vec![], denied_tools: vec!["delete_repo".into()] };
assert!(engine.evaluate_tool_call_with(&allowlist, "read_file").is_allowed()); // allowed
assert!(engine.evaluate_tool_call_with(&allowlist, "delete_repo").is_denied()); // denied
assert!(engine.evaluate_tool_call_with(&allowlist, "unknown").is_denied());     // deny-by-default

Prefer the engine directly (no umbrella)? cargo add ferrumdeck-policy — the import path is still use fd_policy::…. Primitives only: ferrumdeck-core (use fd_core::…). All three are Apache-2.0. (Published as ferrumdeck* because the bare fd-core name is taken on crates.io by an unrelated crate; the Rust import paths are unchanged.)

The 0.8.0 headline feature — the audit trail + out-of-band chain-head checkpoint anchoring (#14: verify_against_checkpoints, CheckpointSigner) — ships as ferrumdeck-audit (import path fd_audit), wired into the umbrella as an optional audit feature so it doesn't bloat the lean engine:

cargo add ferrumdeck --features audit

access path ferrumdeck::audit (e.g. ferrumdeck::audit::CheckpointSigner, ferrumdeck::audit::verify_against_checkpoints). The remaining crates (fd-registry, fd-storage, and the gateway service) are workspace-internal today — they build from a clone but are not published.

What the default omits, and why audit is opt-in. A plain cargo add ferrumdeck gives you the enforcement engine — deny-by-default tool policy, per-run budgets, Airlock RASP — but not the audit hash-chain and signed chain-head checkpoints, which sit behind --features audit. That's deliberate, not an oversight: the audit trail is a separate concern from the in-path allow/deny decision (it pulls in Ed25519 signing and is about after-the-fact tamper-evidence, not stopping the call), so the default stays the lean policy engine most callers reach for. Turn it on when you need tamper-evident record-keeping — e.g. EU AI Act Art. 12/19 logging or Colorado SB 26-189 retention.

Python plane: install from source only. Only the Rust enforcement engine is published to a package registry (crates.io). The Python data-plane packages (fd-runtime, fd-worker, fd-mcp-router, fd-mcp-tools, fd-evals, fd-cli) and the workspace-root ferrumdeck package are not on PyPI — their version numbers are internal workspace versions, not PyPI releases. Install them from a clone with uv: uv sync.

What this proves

FerrumDeck is the control plane, not the agent — the production layer that makes an autonomous agent safe to run: it decides which tools a run may call, kills runs that blow their budget, gates risky actions on a human, and records every decision in an immutable trail. It's built as a credibility artifact for an AgentOps / AI-infrastructure audience.

▶ Run the 5-minute reproducible demo → — one command boots the local stack and verifies, against the real gateway API, the four guarantees below:

• Deny-by-default tool policy — a run may only call tools on its per-agent allowlist; everything else is denied. (POST /v1/runs/{id}/check-tool)
• Budget auto-kill — every run carries a hard token / cost / tool-call / wall-time budget; a breach kills the run and appends a budget.exceeded event. (fd_policy::budget → RunStatus::BudgetKilled)
• Coherence-divergence caught mid-run — when an agent states a blocking fact ("tests failing", "permission denied") and then advances as if it were untrue, the live monitor catches it on the run stream and applies the reversibility ladder (R1–R3). (fd_policy::airlock::coherence)
• Immutable audit trail — every policy, budget, and approval decision is appended to audit_events; the repository exposes no UPDATE/DELETE.
• OTel GenAI spans — and every enforce decision is one — every LLM/tool step emits OpenTelemetry GenAI-semconv spans to Jaeger, and so does every enforcement decision: the tool-policy check writes a gen_ai.tool.call span carrying ferrumdeck.decision = allow|deny|approval|kill, ferrumdeck.reason, ferrumdeck.rung (R1–R3), and ferrumdeck.budget_remaining, so the allow/deny you enforce is the span you query. Naming follows the GenAI-semconv stability opt-in (OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental → execute_tool + gen_ai.operation.name); Rust gateway and Python worker write one schema. (fd_otel::decision · fd_runtime.tracing)

The demo is self-verifying — it asserts each property with jq and exits non-zero on failure, so it works as a smoke test, not a screenshot. For an honest map of what enforces today vs. what's still being wired, see Project Status & Limitations.

Reproduced: indirect-prompt-injection defense-path coverage

Against a vendored AgentDojo-style (arXiv:2406.13352) indirect-injection corpus (25 cases: 17 attacks — off-allowlist tools, RCE payloads, data-exfil destinations — and 8 benign controls), FerrumDeck's deny-by-default tool allowlist + Airlock RASP block:

One-command repro (deterministic, offline, no LLM required):

uv run python -m fd_evals injection-defense --suite injection_defense   # or: make eval-injection-defense

Honest framing: this measures defense-path coverage on a fixed governance profile — the fraction of injected malicious tool calls the policy/RASP layer blocks — not model robustness, and not a general "injection-proof" claim. The corpus is pinned to the real Rust fd_policy RASP by cargo test -p fd-policy --test injection_defense (which runs the actual AirlockInspector over every case), and the fd-evals mirror must agree with it. Small vendored corpus ⇒ a wide CI; the number moves as the corpus grows. No "first"/"best" claim.

Second axis: Agent Security Bench (ASB) + an EU AI Act Art. 50 transparency rule

The injection_defense axis above is AgentDojo (indirect injection). The asb axis adds a second, complementary set: attack classes from Agent Security Bench (arXiv:2410.02644) that AgentDojo does not cover — the Plan-of-Thought (PoT) backdoor, memory poisoning, and direct prompt injection — run through the same governance path plus the R1–R3 reversibility ladder. The point of difference: a backdoored plan that reaches an allowlisted-but-irreversible action (a deploy, an apply_migration) is stopped not by the allowlist but by the R3 rung (irreversible → require_approval, so it never auto-executes). That is graduated enforcement catching an attack a static allowlist would wave through.

Of the 13 ASB attacks, 4 are stopped specifically by the R3 reversibility rung (the PoT-backdoor class), the rest by deny-by-default, the anti-RCE matcher, and the exfiltration shield.

The EU AI Act Article 50 rule (fd_policy::transparency_art50) is a transparency enforcement rule on the same R1–R3 ladder: a governed generative response must carry a human-readable AI disclosure and a machine-readable synthetic-content marker; in enforce mode a response missing either is denied before release (R3), in shadow it is logged (R1).

One-command repro (deterministic, offline, seeded, no LLM):

uv run python -m fd_evals asb --suite asb --seed 0   # or: make eval-asb

Honest framing: the ASB number measures whether the malicious action is stopped from auto-executing on a fixed governance profile — not that FerrumDeck detects the backdoor trigger or the poisoned memory semantically, and not model robustness (no LLM). The Art. 50 rule is a structural check (presence of a disclosure phrase + a machine-readable marker), not a truthfulness or standard-conformance (C2PA/SynthID) judgement. Both are pinned to the real Rust enforcement by cargo test -p fd-policy --test asb_defense — which runs the actual ToolAllowlist + AirlockInspector + reversibility ladder and the real transparency_art50 rule over every case, and asserts the R3 rung is the decisive layer for the PoT class — and the fd_evals.asb mirror must agree. Small vendored corpus ⇒ wide CIs. No "first"/"best" claim.

Live-fire: a drifting run is caught and R-tiered (deterministic, no stack)

examples/demo/coherence-drift.py feeds a deliberately drifting trajectory through the same detection core the live gateway runs on each step, and is self-verifying (exits non-zero if the drift is not caught). Real captured output:

▸ Drifting run — states a blocking fact, then advances as if it were untrue
  ✗ DIVERGENCE [test_failure] confidence=1.00
      stated:       '2 tests failed: assertion error in auth flow'
      then did:     'git_commit: commit: fix auth flow'
      response:     R3 (require_approval)   shadow → record + surface   enforce → GATE run → WaitingApproval
      sse coherence.divergence.detected {category:test_failure, response_rung:R3, gated(enforce):True}
  ✗ DIVERGENCE [permission_denied] confidence=1.00
      stated:       'deploy blocked: permission denied on prod'
      then did:     'deploy: shipping to production, reporting success'
      response:     R3 (require_approval)   shadow → record + surface   enforce → GATE run → WaitingApproval

▸ Coherent control — resolves the fact before advancing (must NOT fire)
  ✓ no divergence — the resolved run is clean

COHERENCE OK ✓  2 divergences fired (both R3), control clean. In enforce mode each R3 gates the run for review.

Against the live gateway the same divergence writes an audit_events row with violation_type=coherence_divergence, emits the coherence.divergence.detected SSE event, and surfaces on the run's Coherence card. It runs shadow by default (records + surfaces, never blocks); set FERRUMDECK_COHERENCE_MODE=enforce to gate an R3 divergence (run → WaitingApproval). This is the enforce-not-observe wedge: FerrumDeck acts on the drift, it doesn't just chart it.

Mapping to the DeepMind AI Control Roadmap (D1–D4 / R1–R3)

Google DeepMind's AI Control Roadmap (2026-06-18, blog) frames agent security on two axes: Detection tiers D1–D4 (set by the model's detection-evasion capability — how hard a misaligned agent is to catch) and Prevention & Response tiers R1–R3 (set by the model's attack-execution capability — how strong the response must be). FerrumDeck already names this ladder in fd_policy::reversibility (anchor deepmind-ai-control-roadmap-r1-r3). To be precise and avoid overclaiming: D1–D4 is an adversary-capability rating, not something a defense tool "is" — FerrumDeck maps onto the R-tier response axis and the roadmap's operational metrics, not onto a self-assigned detection tier.

Response tier → FerrumDeck mechanism (R3 = synchronous block-before-execute; R1 = asynchronous review-after-the-fact):

Detection → the roadmap's live metrics. DeepMind measures a control system by coverage (fraction of agent traffic monitored), recall (fraction of misaligned behaviors caught), and time-to-response:

• Coverage — the CoherenceMonitor observes the run's trajectory event stream (each submitted step), so coverage is the fraction of the run stream it sees. (Honest limit: today it is fed at the gateway step-submission choke point, not the in-loop agentic executor — see Project Status & Limitations.)
• Recall — the divergence pattern is the Strained-Coherence signal (arxiv:2606.07889); the shared cross-plane golden fixture pins detection so recall doesn't silently regress.
• Time-to-response — CoherenceMonitor::observe_event fires mid-run, the instant a divergence completes, before the run finishes — not only in a post-hoc autopsy. That is what moves the same signal from R1 (async) to R3 (synchronous) when enforce is on.

Honest scope. R3 enforcement of a coherence divergence is opt-in (FERRUMDECK_COHERENCE_MODE=enforce) and shadow-by-default — the heuristic is a lexical matcher with a non-zero false-positive rate, so gating is a deliberate choice for high-consequence agents (the roadmap's own point: R1 async review is insufficient for critical-infrastructure agents where a harmful action can't be undone after the fact). The mapping above is machine-checked: examples/demo/coherence-drift.py prints and asserts the R3 → "DeepMind R3: synchronous block-before-execute" label and that the enforce path gates it, exiting non-zero if that ever stops holding. Several advertised layers remain in-progress — see Project Status & Limitations before relying on any tier.

Table of Contents

• What This Proves
  • Mapping to the DeepMind AI Control Roadmap (D1–D4 / R1–R3)
• Overview
• Project Status & Limitations
• Key Features
• Architecture
• Quick Start
• Project Structure
• Components
  • Control Plane (Rust)
  • Data Plane (Python)
  • Dashboard (Next.js)
• API Reference
• Configuration
• Security Model
• Observability
• Evaluation Framework
• Development
• Deployment
• License

Overview

FerrumDeck solves the critical challenge of running AI agents safely in production. While LLMs are probabilistic and unpredictable, production systems require deterministic governance, audit trails, and budget controls.

The Problem

• AI agents can make costly mistakes (token spend, wrong tool calls)
• Prompt injection attacks can bypass safety measures
• No visibility into what agents are doing in production
• Difficult to reproduce and debug agent failures
• Compliance requirements demand audit trails

The Solution

FerrumDeck provides a dual-plane architecture:

Project Status & Limitations

FerrumDeck is an early-stage / alpha project, built primarily by a single maintainer. It is a real, working control plane — but it is not yet production-hardened. This is an honest map of what enforces today vs. what is scaffolded or on the roadmap, so you can evaluate it without surprises.

Implemented and enforced (covered by the Rust test suite):

• Deny-by-default tool policy, per agent. The gateway evaluates every tool call against the run's agent allowlist (allowed / approval-required / denied tiers) — not a process-global default.
• Budget enforcement, per run / per agent. The auto-kill and the cost forecast evaluate against the run's effective budget (per-run config.budget override → agent-version caps → engine default).
• Tenant isolation. Project-scoped access is gated by a DB-backed project → workspace → tenant ownership check; unknown project or tenant mismatch is denied.
• Airlock RASP at the gateway tool-policy check (POST /v1/runs/{id}/check-tool): all five layers run here, in shadow or enforce mode — the anti-RCE pattern matcher, the financial/velocity circuit breaker, and the data-exfiltration + credential-DLP shield on every call, plus the schema-drift guard (validates tool_input against the tool version's registered input schema) when the tool has a registered version and the behavioral-drift monitor (per-agent rolling z-score on cost) when the run's agent is known. The gateway attaches the guard + monitor at boot (state.rs) and threads tool_version_id / agent_id into the inspection context (check_tool_policy). The schema-drift guard is seeded at boot from the tool_versions table and refreshed live on every tool registration (create_tool calls guard.upsert), so a version registered after boot is drift-checked without a restart. A tool version with no compiled schema is fail-open by default; FERRUMDECK_SCHEMA_DRIFT_FAIL_CLOSED=true flips that case to deny-by-default.
• Enforcement on the agentic execution path. The Python worker's in-loop agentic executor authorizes every tool call against the control-plane check-tool endpoint before it runs: allow → execute, deny → refuse, requires_approval → do not execute (the run is gated pending approval). The local allowlist is only a cheap pre-filter, never the final authority, and the path fails closed — if the control plane is unreachable the call is refused (configurable via AGENTIC_FAIL_CLOSED, default on). (Previously this loop only checked a local allowlist and executed approval-required tools anyway; fixed 2026-07-27.)
• Append-only audit trail for policy, budget, approval, routing, and promotion decisions (the repository exposes no UPDATE/DELETE).
• Coherence-divergence monitor, wired live at the gateway run stream. As each step is submitted, the run's trajectory is fed to the CoherenceMonitor; a stated-blocking-fact → contradicting-closure-action divergence surfaces mid-run through the same airlock.violation_detected audit path, is persisted on the run row (coherence_divergence_flagged), and emitted on the completion span. A reliability signal — it never blocks a tool or kills a run.

Scaffolded / not yet wired end-to-end — do not rely on these yet:

• Trace→signal loop (HarnessX). The harness-suggestion governance endpoints (/v1/harness-suggestions*) and the training-signal export (POST /v1/runs/{id}/training-signal, redacted server-side via the audit redaction path) are implemented, unit-tested, and wired into the dashboard. The evals dashboard read path is no longer stubbed: /api/v1/evals/* serves the gateway's real on-disk reports, mapped onto the dashboard's run contract, and returns an explicit 501 (never an empty list) when no report store is reachable. Two defects on that path were fixed at once — the gateway parsed only the offline-benchmark file-naming convention, so every safe-PR smoke and regression report was silently dropped before it reached the dashboard, and the BFF proxied the gateway's field names verbatim, so even the runs that did arrive rendered as blank cells. Not yet verified against a live stack (Docker was unavailable when this landed): the projection is covered by unit tests that read the real committed reports through the same code path the handler uses, but the live HTTP round-trip is unconfirmed. Dispatching a run from the dashboard remains unimplemented — the store is read-only committed records, so #7 stays open for the live verification and the dispatch path. (The eval numbers themselves are not ungated: the deterministic governance suites + real-engine pins run on every push + PR via ci.yml's eval-regression job.) Approving a suggestion records the decision; it never auto-applies a policy/allowlist/budget change.
• Eval gating — deterministic suites gate PRs; the LLM-backed nightly does not yet. docs/eval-health.md is generated from the committed report files on every nightly run and shows, per eval, the last run date, pass/fail, score, consecutive-pass streak and assertion coverage. An eval that has never passed is labelled NEVER PASSED in its own row. Read that page before trusting any eval-gating claim here — it is the evidence, and it opens by stating in prose what the safe-PR numbers mean. docs/eval-verdicts.md carries the companion judgement: one row per eval saying whether its score is evidence about the agent or about the harness, with the reproduction command. In short: the safe-PR suites do not measure safe-PR agent quality. Their dataset expects files changed, a PR opened and tests passing against example/project, which does not exist and which this control plane never clones — so the suites were rescoped to assert what is genuinely observable here (policy decisions, budget compliance, non-degenerate output). The expectations no scorer reads are reported on every run rather than averaged past. There is no eval in this repo that measures whether the agent writes good pull requests, and nothing here should be read as claiming there is.
• Audit tamper-evidence — detectable up to the last checkpoint, not tamper-proof. The log is append-only (repo API with no UPDATE/DELETE path and the trg_audit_events_append_only trigger, migration 20260719000001) and hash-chained: migration 20260801000001 adds prev_hash/record_hash/chain_seq, each row's SHA-256 commits to its predecessor over a canonical encoding (rust/crates/fd-audit/src/chain.rs), and AuditRepo::verify_chain catches any insertion, deletion, or edit within a tenant's chain. The chain alone left one honest gap — a privileged actor who rewrites the entire tail (dropping the trigger, recomputing every downstream hash) can produce a self-consistent chain, because they hold every input. That gap is now closed by signed head checkpoints (rust/crates/fd-audit/src/checkpoint.rs): a small (tenant_id, chain_seq, record_hash, checkpointed_at) record, signed with an Ed25519 key that is not the database's, is appended to an out-of-band sink (FileCheckpointSink ships; the CheckpointSink trait takes object storage or a transparency log later). verify_against_checkpoints proves the chain has not been rewritten past the most recent checkpoint — because a record's hash transitively commits to its whole prefix — and names the checkpoint it verified against. The guarantee, stated exactly: tampering is detectable up to the most recent checkpoint; records after it keep only the in-chain guarantee (that window is reported, not hidden), a missing checkpoint degrades to the in-chain guarantee and says so rather than silently passing, and this is detection, not prevention — not tamper-proof, and only as strong as the sink being genuinely out-of-band and the signing key off-box (a file sink on the DB host is a weak anchor). Shipped for #14; a robust remote sink + off-host key custody is the remaining hardening.
• Multi-tenant SaaS hardening. Tenant isolation is enforced, but there is no dashboard auth/session layer, no SSO/RBAC, and no API-key self-service — treat the dashboard + gateway as a trusted-operator deployment for now.
• Realtime run stream (SSE). Until the gateway→BFF push lands (#5) the dashboard's realtime channel carries heartbeats only — governance events are read from the polled run endpoint, not pushed. The BFF can emit synthetic events for wire-shape development behind FERRUMDECK_SSE_MOCK_EVENTS, but that flag is OFF by default in every environment, so a fabricated enforcement verdict (a synthetic R3 gate, a made-up policy decision) can never reach an operator's console.

Testing caveat. The unit/lint suites (cargo test --workspace, clippy, ruff, jest) pass and gate CI. The tests/security, tests/chaos, and tests/e2e suites require a live stack (make dev-up) and skip without it. Hardening them to assert behaviour (not liveness) is in progress (#6): test_airlock.py (RCE, raw-IP exfil, credential DLP), test_policy_engine.py, test_input_validation.py (command injection), and test_owasp_llm.py (deny-by-default tool policy) now assert the actual enforcement decision — the specific violation_type + risk band, or allowed=false — and the tests/chaos suite's simulated no-ops are now explicit skips naming the fault-injection plumbing they need, not green tautologies. The Airlock layer decisions also have a runnable, no-Docker backbone in rust/crates/fd-policy/tests/airlock_decisions.rs + airlock_layers_fire.rs.

One headline case per suite has since been converted to a behavioural assertion: tests/security asserts that a denied tool call leaves no side effect and a durable record (with a negative control, so it cannot pass against an engine that records every call as blocked); tests/chaos asserts that a policy decision taken before a Postgres outage is identical after recovery; tests/e2e asserts that an exhausted cost budget refuses the next call rather than logging and continuing. None of the three has been observed passing against a live gateway — they are written against the real routes and skip cleanly without a stack, but the assertions themselves are unverified until someone runs them with make quickstart up, and a dedicated live-stack CI job is still the missing infra piece on #6. Fixing the two conftests that made this visible is a finding in its own right: tests/chaos and tests/e2e probed /health/live, a route the gateway does not serve, so both suites skipped unconditionally against any stack and CHAOS-001 has never once executed. Still liveness-only and deferred on #6: the rest of tests/e2e, the remaining tests/security/test_audit_trail.py cases, and the stateful/obfuscation cases (velocity, coherence, base64/unicode evasion) — do not read those as proof that a given attack is blocked.

Automated test coverage. The CI-gating unit/lint suites total 1,954 tests, re-derivable with make claims-recount: Rust 749 (cargo test --workspace -- --list), Python unit 532 (pytest over the four python/packages/*/tests the CI unit job runs), frontend 623 (jest), and API-contract 50 (pytest tests/api). The live-stack suites — tests/security (78), tests/chaos (14), tests/e2e (43) — need make dev-up, skip without it, and (per the caveat above) are still being converted from liveness to behaviour, so they are excluded from that headline; tests/integration (81) is likewise live-stack and non-gating. These counts live in the single source docs/feature-status.yml and are held to the README by the claims-integrity CI check (make check-claims).

The known gaps above are tracked in the open on the **[roadmap](https://github.com/sattyamjjain/ferrumdeck/blob/main/ROADMAP.m