e2e-skills gives AI coding agents four focused workflows for Playwright and Cypress E2E work: generate Playwright coverage, review existing Playwright/Cypress specs, debug failed Playwright reports, and debug failed Cypress reports. It also includes a deterministic scanner for the mechanically detectable subset of the review catalog.

Need	Skill	Result
Generate new Playwright coverage	`playwright-test-generator`	Explored, approved, reviewed Playwright specs
Review existing Playwright/Cypress tests	`e2e-reviewer`	Verified P0/P1/P2 findings with concrete fixes
Debug a failed Playwright run	`playwright-debugger`	F1–F15 root cause, evidence, and fix
Debug a failed Cypress run	`cypress-debugger`	F1–F15 root cause, evidence, and fix
Run a deterministic local scan	`skills/e2e-reviewer/scripts/scan.sh`	Mechanical candidates without target-project packages

The generator starts with coverage-gap analysis and live-browser exploration, then generates after scenario approval and verifies each candidate. The debuggers start from failed-run artifacts and return a classified root cause, supporting evidence, and a concrete fix.

False-green detection is one important part of the review workflow, not the bundle's entire purpose. Fixes based on e2e-reviewer findings have been merged through 14 merged upstream PRs, including Storybook, SvelteKit, code-server, Strapi, Carbon Design System, Ghost, and MUI X.

In code-server, a committed it.only silently disabled eight tests for seven months. One skipped test was already broken while CI remained green.

Executable example: React optimistic-write proof shows why optimistic UI needs request and persistence proof, not just visible state.

Merged upstream fixes

e2e-reviewer findings have contributed to 14 merged upstream PRs. These self-selected cases show practical use and let readers inspect the fixes; they are not a representative validation sample or an accuracy estimate.

Repository	PR	Pattern fixed
Storybook	storybookjs/storybook#34141	Missing `await` on Playwright assertions
code-server	coder/code-server#7845	Focused test leak, matcher-less `expect`, discarded visibility read
Strapi	strapi/strapi#26630	Discarded navigation/state checks
SvelteKit	sveltejs/kit#16068	Floating Playwright assertions
Carbon Design System	carbon-design-system/carbon#22564	Locator truthiness replaced with web-first assertions
Ghost	TryGhost/Ghost#28712	Promise-valued disabled-state assertion
Cal.com	calcom/cal.diy#28486	Weak assertion patterns in E2E flow
Bruno	usebruno/bruno#8317	Assertion and wait reliability fixes
Qwik	QwikDev/qwik#8777	Locator/handle existence checks
Element Web	element-hq/element-web#32801	Locator null-check style assertions
MUI X	mui/mui-x#22982	UI handle checks replaced with state assertions
module-federation/core	module-federation/core#4826	Redundant blanket `uncaught:exception` suppression in a Cypress spec
FiftyOne	voxel51/fiftyone#7851	Locator-defined check replaced with a visible duplicate-name error assertion
Rancher Desktop	rancher-sandbox/rancher-desktop#10557	`not.toBeNull()` locator checks replaced with visible WSL integration-name assertions

See a false-green test

A false-green test passes whether or not the behavior it names works. It is not a flaky test: a flaky test fails sometimes, so retry dashboards and flake analytics eventually see it. A false-green test never fails — including when the product is broken — so pass/fail monitoring alone will not surface it.

This Playwright test looks reasonable but proves only that Locator objects were created:

import { expect, test } from '@playwright/test';

test('shows the welcome message', async ({ page }) => {
  await page.goto('/dashboard');
  expect(page.getByText('Welcome back')).toBeDefined();
  expect(page.locator('.user-badge')).not.toBeNull();
});

A useful test verifies visible behavior and can fail when that behavior breaks:

- expect(page.getByText('Welcome back')).toBeDefined()
+ await expect(page.getByText('Welcome back')).toBeVisible()

The bundled scanner catches the false-green assertions without project configuration. A shortened run shows the actionable lines:

$ /bin/bash -p skills/e2e-reviewer/scripts/scan.sh tests/

[P0] #4f Locator always-true assertion (truthy/defined/not-null) (2 hits)
  .../tests/login.spec.ts:5:  expect(page.getByText('Welcome back')).toBeDefined();
  .../tests/login.spec.ts:6:  expect(page.locator('.user-badge')).not.toBeNull();

Summary: 2 total hit(s), 2 P0

Because this run reports P0 findings, scan.sh exits 1.

eslint-plugin-playwright also flags this shape via no-unnecessary-assertions; enable it so commit-time lint owns what it can and the scanner fills the gaps.

Prove the test can fail

A good matcher is not enough; the test must go red when behavior breaks. For each candidate, V2 inverts its primary assertion and V3 injects an evidenced fault in an approved scratch copy. It must fail at the predicted line with the predicted mismatch. The source stays byte-identical; timeouts, browser crashes, and config errors do not count, and unsafe probes are CANNOT_VERIFY.

Install and try it

Claude Code

Install from the plugin marketplace:

/plugin marketplace add voidmatcha/e2e-skills
/plugin install e2e-skills@voidmatcha

Or install copied skills through the pinned cross-agent CLI:

npx --yes [email protected] add voidmatcha/e2e-skills --skill '*' -g -a claude-code

Codex

Install the four skills into ~/.agents/skills/:

npx --yes [email protected] add voidmatcha/e2e-skills --skill '*' -g -a codex

For Codex delegation, e2e-reviewer, playwright-debugger, and cypress-debugger can use native roles or their equivalent inline fallbacks. playwright-test-generator has a stricter V6 boundary: without a distinct fresh-context reviewer it reports CANNOT_VERIFY and PARTIAL/BLOCKED. Source checkouts also include optional native agents under .codex/agents/; contributors can see AGENTS.md for the packaging boundary.

Alternative Codex plugin marketplace path:

codex plugin marketplace add voidmatcha/e2e-skills
codex plugin add e2e-skills@voidmatcha

Other agents

Install globally for every host supported by the skills CLI:

npx --yes [email protected] add voidmatcha/e2e-skills -g --all

To target one host, replace --all with -a <agent>; see the supported agents. The commands pin the reviewed CLI release instead of executing an unreviewed newer version.

Manual Claude Code checkout

Keep the checkout outside ~/.claude/skills/, then link each public skill directory:

git clone https://github.com/voidmatcha/e2e-skills.git "$HOME/.claude/e2e-skills"
mkdir -p "$HOME/.claude/skills"

for skill in playwright-test-generator e2e-reviewer playwright-debugger cypress-debugger; do
  ln -s "$HOME/.claude/e2e-skills/skills/$skill" "$HOME/.claude/skills/$skill"
done

The links fail rather than replacing existing same-named skills. Run /skills in Claude Code and confirm all four names appear.

First prompts

Review my Playwright tests in tests/e2e with e2e-reviewer.

Generate Playwright E2E coverage for apps/web/e2e.

Debug the failed Playwright report in playwright-report/.
Debug the failed Cypress report in cypress/reports/.

Scope and limits

Use this bundle to generate or review E2E tests and to diagnose failed Playwright/Cypress runs. Use it alongside the application and its real E2E suite, not instead of them; it is not a general lint preset or a framework-agnostic test tool. Playwright and Cypress are supported; new-test generation currently targets Playwright only.

The bundled shell scripts and artifact readers target macOS/Linux shells. Windows users should run them through WSL and keep scan/report artifacts inside the WSL filesystem.

A green generated test is not enough: it may assert a Locator or Promise, observe state unrelated to the behavior named by the test, or leave its primary assertion non-load-bearing. The generator therefore treats every new spec as a candidate until all applicable V1–V6 verification passes.

How the review works

Generating valid test code is not the same as generating a test that fails when the product is wrong. The workflow separates mechanical detection from semantic judgment:

The scanner finds deterministic candidates such as Locator truthiness, focused tests, missing await, and blanket error suppression.
e2e-reviewer reads test names, actions, assertions, helpers, Page Objects, fixtures, and configuration before confirming a finding.
Findings use stable pattern IDs and P0/P1/P2 severity so fixes and regressions remain comparable.
After a fix, the workflow reruns the scanner and the project's approved E2E or lint command.

A scanner match is a candidate, not a verdict. Cross-file findings such as missing authentication, optimistic UI without call proof, name/assertion mismatch, and fixtures blocked by render guards require semantic review.

Evidence and limits

The current evidence supports a narrow claim: the project has behavior-backed development evidence and 14 merged upstream fixes, but it does not claim generalized reviewer accuracy.

Browser fault injection completed 36/36 Playwright/Cypress cells.
The exact reviewer benchmark covers 12 proven false-green cases and 12 clean guards; ten fault cases are byte-identical operator mutants.
Independent robustness gates v4, v5, v7, and v8 failed their preregistered criteria. V6 and v9 were not run, and v10 is frozen but not run.
The debugger protocol provides a replayable 30-case synthetic corpus, but no independently established debugger accuracy is claimed.

See benchmark status for scores, failed gates, superseded runs, and claim boundaries. The research evidence ledger audits 59 external sources instead of treating adjacent unit-test or custom-agent studies as measurements of this project.

E2E review catalog

The catalog contains 24 stable Playwright/Cypress test smells. The most common false-green shapes include Locator truthiness, missing assertions, swallowed errors, focused tests, missing authentication, and optimistic UI checks without network proof. See the full taxonomy and rationale.

Some patterns need the application in scope, not just the tests. #22 optimistic UI is the clearest case: whether a click issues a mutation cannot be decided from a spec, so on a repository that holds tests alone the review reports nothing for it rather than guessing. That is deliberate false-positive control, and it is why the executable example ships with a component.

24 Patterns Detected — Grouped by Severity

P0 — Must Fix (silent always-pass)

Tests pass when the feature is broken. No real verification is happening.

#	Pattern	Before	After
1	Name-assertion mismatch	Name says "status" but only checks `toBeVisible()`	Add assertion for status content, or rename to match actual check
2	Missing Then	Cancel action, verify text restored — but input still visible?	Verify both restored state and dismissed state
3	Error swallowing	`try/catch` in spec, `.catch(() => {})` in POM	Let errors fail; remove silent catch from POM methods
3b	Cypress `uncaught:exception` suppression	`cy.on('uncaught:exception', () => false)` blanket-swallows app errors	Scope handler to specific known errors; re-throw unknown errors
4	Vacuous or retry-weakening assertion (P0/P1)	P0: invariant predicates and Locator truthiness. P1: weak attachment proof; one-shot values/URL; zero-timeout retry/deadline hazards; unproven absence; ARIA snapshots that omit a promised accessible name	Use meaningful bounds and web-first auto-retrying assertions; prove presence before absence and keep promised accessible names load-bearing
5	Bypass patterns (5a P0, 5b P1)	`if (await el.isVisible()) { expect(...) }`; `{ force: true }` without comment	Always assert; move env checks to `beforeEach`; add `// JUSTIFIED:` to force:true
7	Focused test leak	`test.only(...)` committed — CI runs one test, silently skips the rest	Delete `.only`; use `--grep` or `--spec` for local focus
8	Missing assertion	Discarded locator/boolean is the scenario's only verification	Add `await expect(locator).toBeVisible()`; skip #8 when independent verification/failure evidence already exists
12	Missing auth setup	With no login/`storageState`/auth fixture, a protected-route spec passes because its generic assertion also matches the login/wrong surface	Add `beforeEach` login, configure `storageState`, or use an auth fixture; do not classify a normal auth-caused failure as P0

P1 — Should Fix (poor diagnostics / wastes CI time)

Tests work but mislead developers, waste CI time, or set up future regressions.

#	Pattern	Before	After
6	Raw DOM queries	`document.querySelector` in `evaluate()`	Use framework locator/query APIs (`locator` / `cy.get`)
9	Hard-coded sleep	`waitForTimeout(2000)` / `cy.wait(2000)` / `waitForLoadState('networkidle')`	Rely on framework auto-wait; use condition-based waits
10	Flaky test patterns	`items.nth(2)` without comment; `test.describe.serial()`; unscoped accessible-name substring (10c); Cypress async callbacks, assigned `cy` commands, or continued action chains (10d–10f)	Use stable/scoped locators and self-contained tests; keep Cypress work in its command chain, do not assign Chainables as values, and re-query after actions
13	Inconsistent POM usage	POM imported but spec uses raw `page.fill`/`page.click` for POM-owned actions	Route all interactions through the POM so UI changes update in one place
14	Hardcoded credentials	`loginPage.login('demo-admin', '<literal-password>')` in test code	Use `process.env.TEST_USER`, Playwright config secrets, or test data fixtures
15	Missing `await` on `expect()`	Async Locator/Page web-first matcher Promise is not sequenced or observed; rejection usually surfaces later with worse attribution	`await` or return the matcher Promise; sync value matchers are excluded
16	Missing `await` on action	Actionability, action ordering, or navigation can race later work; rejection usually surfaces later with worse attribution	`await` or return the action Promise
17	Discouraged direct Page selector API	Selector-based `page.click`, `page.fill`, and related Page actions skip the Locator layer	Use Locator actions for composition, strictness, reuse, and clearer failures
18	`expect.soft()` overuse	Critical soft assertions run before a hard scenario gate, so dependent work continues after a broken prerequisite	Hard-gate the primary state first; use `soft` only for independent details
19	Module-level mutable state in test code	`let seq = 0;` or a mutated `const cache = new Map()` at column 0 in a test utility — persists across tests in a long-lived worker and collides across parallel workers	Drop the counter; derive uniqueness from `Date.now()` + `Math.random().toString(36).slice(2, 8)`, or move state into `test.beforeEach`
20	Unmocked real-backend writes	Signup/checkout spec reaches shared or persistent state with no controlled test boundary	Stub the write or prove a disposable container, rollback fixture, isolated tenant/database, or equivalent controlled backend
22	Optimistic UI without call proof	Like-toggle test asserts `aria-pressed` flip — UI updates optimistically, passes with the POST deleted	Pair UI assertion with `page.waitForRequest()` (armed before the click) or a route-hit flag

P2 — Nice to Fix (maintenance / robustness)

Weak but not wrong — addressed when refactoring.

#	Pattern	Before	After
11	YAGNI + Zombie Specs	`clickEdit()` never called; unjustified empty wrapper class; entire spec duplicated by another	Delete unused members and zombie specs; inline single-use helpers only when that clearly removes meaningless indirection
21	Manually-captured session-file dependency	`storageState: 'auth/member.json'` produced only by a manual capture script — absent on CI, silently expires	Regenerate session programmatically (API-login helper or `setup` project); manual files only as a cache with a programmatic fallback
23	Fixture ignores render guards	Liked-tab fixture seeds `liked: false`; the card component `return null`s every item — empty UI looks like infra flake	Read the item component's early returns/filters before seeding; seed fields to pass every guard for the view under test

Failure debugging

Both debuggers use the same stable F1–F15 root-cause taxonomy. Playwright accepts playwright-report/, HTML reports, trace.zip, screenshots, and bounded GitHub Actions artifacts. Cypress accepts mochawesome or JUnit reports, screenshots, videos, and bounded CI artifacts.

#	Category	Signals
F1	Flaky / Timing	`TimeoutError`, passes on retry
F2	Selector Broken	`locator not found`, strict mode violation
F3	Network Dependency	`net::ERR_*`, unexpected API response
F4	Assertion Mismatch	`Expected X to equal Y`, subject-inversion
F5	Missing Then	Action completed but wrong state remains
F6	Condition Branch Missing	Element conditionally present, assertion always runs
F7	Test Isolation Failure	Passes alone, fails in suite
F8	Environment Mismatch	CI vs local only; viewport, OS, timezone
F9	Data Dependency	Missing seed data, hardcoded IDs
F10	Auth / Session	Session expired, role-based UI not rendered
F11	Async Order Assumption	`Promise.all` order, parallel race
F12	POM / Locator Drift	DOM structure changed, POM not updated
F13	Error Swallowing	`.catch(() => {})` hiding actual failure
F14	Animation Race	Content not yet rendered, or a transient element removed before it is observed
F15	Hydration Race	Action succeeds but has no effect — SSR page not yet hydrated; fails at the next assertion

The debuggers classify product regressions separately from brittle tests and return evidence plus a concrete fix. They do not diagnose an application or backend without a failing Playwright or Cypress test artifact.

Standalone scanner

Run the deterministic mechanical layer directly:

/bin/bash -p skills/e2e-reviewer/scripts/scan.sh path/to/tests

The scanner requires a PCRE2-capable rg and Python 3. Python creates and validates NUL-safe candidate identity records so candidate drift or malformed records fail closed; this mandatory bookkeeping is separate from optional Tier 2 AST tooling. It does not execute target-controlled ESLint binaries, plugins, parsers, or configuration by default, and it does not download tools by default. E2E_SMELL_ALLOW_PROJECT_ESLINT=1 opts a trusted checkout into project ESLint execution; E2E_SMELL_NO_ESLINT_DOWNLOAD=0 and E2E_SMELL_NO_AST_GREP_DOWNLOAD=0 separately opt into pinned downloads. Set E2E_SMELL_DISABLE_AST_GREP=1 when a portability check must ignore preinstalled host binaries.

Read boundary.

Bundled checks report source beneath the requested path. Framework provenance resolution may also read relative fixture/support imports elsewhere in the containing project.

Bundled checks read .ts, .js, .tsx, .jsx, .mts, .mjs, .cts, and .cjs sources.

Tier 3 is the bundled fallback. Optional ESLint and ast-grep tiers add precision but do not replace semantic review. The scanner exits 2 on infrastructure or filesystem errors rather than reporting a false clean result. See SECURITY.md for the trust and network boundary.

How this differs from ESLint plugins

eslint-plugin-playwright and eslint-plugin-cypress are good every-commit baselines for syntactic rules. e2e-skills adds two different layers:

A secure-default scanner that does not run the target project's lint stack unless explicitly enabled
Semantic review for findings that need test intent or cross-file context

A linter can catch a direct Locator truthiness assertion or missing await. It cannot decide whether a test named “shows a duplicate-name error” ever checks the error, whether a protected-route test forgot authentication, or whether an optimistic UI assertion proves the backend request happened. Use the plugins for continuous linting and e2e-reviewer for test trustworthiness.

Frequently asked questions

How do I find Playwright or Cypress tests that pass but test nothing?

The e2e-reviewer skill reviews all 24 catalog patterns with stable IDs and P0/P1/P2 severity. Its standalone scan.sh scanner covers only a deterministic mechanical subset. Scanner matches are candidates, not final findings; the skill checks intent and surrounding code before reporting a verdict.

The debuggers classify failures against the stable F1–F15 taxonomy. They and the generator execute target-controlled code only after you trust the repository and approve the exact command, including its environment and flags.

For non-public benchmark runs, --isolation-wrapper is a required hook, not proof of isolation. Continuous integration (CI) validates the wrapper contract but does not attest filesystem, process, or network isolation.

Point e2e-reviewer at the relevant spec directory. It combines deterministic candidates with semantic review before returning findings.

Does this replace Playwright or Cypress test execution?

No. Run the application and its real E2E suite after every change. This bundle reviews test quality, generates Playwright coverage, and diagnoses existing failures; it is not a test runner.

How do I review AI-generated E2E tests?

Point e2e-reviewer at the generated specs before merge. It checks whether each test proves its stated user-visible outcome, then separates deterministic scanner candidates from context-dependent findings.

Does it support Cypress as well as Playwright?

Review and failure debugging support both frameworks. New-test generation currently supports Playwright only. Cypress debuggers accept mochawesome and JUnit reports.

Can it debug tests that fail only in CI?

Yes, when you provide the local report artifacts or a supported GitHub Actions run. The debugger separates environment, timing, selector, data, authentication, and product-regression causes using the F1–F15 taxonomy.

Which AI coding agents are supported?

Claude Code, Codex, and the 55+ hosts supported by the skills CLI can load the public SKILL.md contracts. Optional host-specific agent files improve delegation where available, but the public skills remain usable without them.

Detailed documentation

Planned work includes cross-model convention consistency and stronger deterministic detection. No roadmap item is described as shipped before its dedicated verification passes.

Contributing

Bug reports, false-positive guards, new anti-patterns, and translations are welcome. Start with CONTRIBUTING.md for setup and verification requirements. Cross-agent maintenance contracts live in AGENTS.md.

E2e Skills

At a glance

README