Tripwire

Does your skill trip on the right prompts?

Lint, real activation-coverage probing, and CI checks for the SKILL.md files that power Agent Skills.

The idea

A skill's description is the only signal the agent uses to decide whether to invoke it. It isn't documentation — it's routing logic, and it fails silently in both directions: too narrow and the skill never fires on its own use cases; too broad and it hijacks unrelated prompts. You can't lint your way to knowing which happens — that's a behavioral property, so you have to run the skill.

Tripwire does exactly that. It lints your SKILL.md against best-practice rules, then probes activation coverage by generating a prompt matrix and running real agent sessions to see what actually fires — surfacing description bugs, coverage gaps, and false positives before they ship.

When you don't need this: if a skill is purely personal — you're the only one who'll ever load it — you'll notice a misfire the next time you use it. Tripwire earns its keep once a skill ships to other people's agents, where a silent gap is invisible until someone reports "it just didn't do the thing."

Quick start

npm install -g tripwire-skills

# Zero-config entry — scaffolds .github/workflows/tripwire.yml and lints your skill
tripwire init ./skills/brainstorming/

# Static best-practice lint (instant, offline, free)
tripwire lint ./skills/brainstorming/SKILL.md

# Probe which prompts actually activate it, then commit the scenarios it writes
tripwire analyze ./skills/brainstorming/

# In CI: rerun those exact scenarios deterministically, no re-probing
tripwire test ./skills/brainstorming/

Prefer not to install anything? The lint engine runs live in your browser at tripwire.bharath.sh — paste a SKILL.md and see it graded instantly.

The GitHub Action

Gate skill changes in CI the same way you gate lint and tests. Add one workflow file:

# .github/workflows/tripwire.yml
name: Tripwire
on: pull_request
jobs:
  skills:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      pull-requests: write   # for the summary comment
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0       # needed to diff the PR
      - uses: bharath31/tripwire@v1
        with:
          probe: true                                       # also run coverage checks
          anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }}  # your key, your runner

On every PR it lints changed skills (always), probes activation coverage (when a key and committed scenarios are present), annotates the exact lines on the diff, and posts a sticky summary comment. The probe runs in your CI using your API key — it never leaves your runner. Without a key (e.g. fork PRs), the probe skips with a notice and lint still gates the PR.

Command reference

analyze is a deliberate local step (it calls real models). It writes a tripwire-scenarios.yaml you commit alongside the skill; the Action and tripwire test then rerun those exact scenarios deterministically.

Going deeper

analyze and test accept --agent <claude|gemini|codex> (default claude) to probe activation against a different agent CLI on the same SKILL.md. Confidence differs by agent:

tripwire analyze ./skills/brainstorming/ --agent=gemini

Two skills can each lint clean and still collide at runtime. tripwire conflicts scans a whole directory and reports name collisions (two files sharing a name — a hard error) and description overlap (two skills sharing enough trigger vocabulary that a prompt could plausibly fire either — an advisory warning).

tripwire conflicts ./skills/
#   → --threshold 0.0-1.0 tunes how much shared vocabulary counts as overlap (default 0.3)

Static and free — no API key. A behavioral probe (does an ambiguous prompt pick the wrong skill) is a natural next step on top of this.

A tripwire-scenarios.yaml committed in March isn't guaranteed to still pass in June — model updates can shift which prompts activate a skill with no code change to explain it. tripwire test-all reruns every skill's committed scenarios in one pass and reports any that now disagree with their baseline:

tripwire test-all ./skills/

tripwire init --drift scaffolds a scheduled workflow (.github/workflows/tripwire-drift.yml, weekly) that runs this on a timer, writes the report to $GITHUB_STEP_SUMMARY, and fails the run when a skill has drifted — the same signal GitHub already uses to notify you of scheduled-workflow failures.

Everything above answers "did the skill activate." tripwire eval answers "once activated, did it do the right thing," with author-written cases in tripwire-evals.yaml:

skillName: brainstorming
cases:
  - name: asks a clarifying question before proposing a solution
    prompt: help me build a new dashboard feature
    assertions:
      - type: contains
        value: "?"
      - type: not_contains
        value: "```"
    rubric: The response should ask about scope or requirements before writing any code.

Each case runs two independent checks: assertions (contains / not_contains — free, deterministic, no key) and an optional rubric (a natural-language bar, graded by an LLM judge, priced like analyze). A rubric case is skipped with a clear notice — never silently ignored — when no ANTHROPIC_API_KEY is set; assertion-only cases still run.

tripwire eval ./skills/brainstorming/

This is the newest, least-established surface here — assertions are solid, but a rubric is only as good as the rubric you write.

For a team standardizing skills across an org, drop this in the same tripwire.yaml the probe config uses (resolved per-skill-directory):

extends: tripwire:recommended   # the only preset today — every built-in rule at its default level
rules:
  no-code-example: off          # off | warning | error — explicit rules always win over a preset
  description-use-when: warning
plugins:
  - ./org-rules.mjs             # plain JS, resolved relative to this tripwire.yaml

A plugin exports plain objects — no dependency on tripwire's own types:

// org-rules.mjs
export const rules = [
  {
    id: 'org-requires-owner',
    defaultLevel: 'error',
    check: (skill) => (skill.frontmatter.owner ? null : 'org policy: every skill needs an `owner` field'),
  },
];

A malformed custom rule is skipped with a warning rather than crashing the run. This applies everywhere lint() runs: tripwire lint, analyze, test, test-all, and the Action.

What it checks

Lint (static, free): name present and kebab-case · description present, starts with "Use when", ≤ 1024 chars, and doesn't summarize a workflow · body isn't a stub (no placeholder text, has an example, meets a length floor).

Coverage probe (real sessions): generates prompts across four zones and checks what actually fires —

A scenario whose real behavior disagrees with its expectation is a regression — a gap (didn't fire when it should) or a false positive (fired when it shouldn't).

Coverage badge

A live badge — no stored state; it fetches your SKILL.md from GitHub and lints it fresh on every request (cached ~5 min at the edge):

![tripwire](https://tripwire.bharath.sh/api/badge?repo=owner/repo&path=path/to/SKILL.md&branch=main)

repo is required (owner/repo); path defaults to SKILL.md at the repo root; branch defaults to main. An unrecognized repo renders a grey "unknown" badge rather than failing — a badge should never break your README's render. (The one at the top of this file points tripwire at its own skill.)

Also ships as

• A skill — copy skills/tripwire/SKILL.md into your own .claude/skills/ and the agent lints every SKILL.md you write as you author it, recommending the CLI and the Action at the right moments (never spending API money without asking).
• A VS Code extension (early scaffold) — vscode-extension/ wires the same lint engine to inline squiggles. It builds, type-checks, and packages into a real .vsix, but isn't published to the Marketplace yet.

How it works

Activation isn't visible in claude -p text output — Tripwire runs claude -p "<prompt>" --output-format stream-json --verbose and detects the Skill tool-use event for the skill under test. In CI, each changed skill is staged at .claude/skills/<name>/SKILL.md so the agent can load and activate it.

Development

npm install
npm test              # run the suite (288 tests)
npm run build         # build the CLI (dist/cli.js)
npm run build:action  # bundle the GitHub Action (action-dist/index.js)
npm run build:web     # bundle the browser playground

The landing page (web/) and its Cloudflare Pages Functions (functions/) auto-deploy to tripwire.bharath.sh on push to main. The README banner is a Remotion composition in banner/ — cd banner && npx remotion render Banner out/banner.gif --codec=gif regenerates assets/banner.gif.