Claude Code skills: build a /smoke-test skill that clicks through your web app in a real browser

Install Playwright CLI and its agent skill, then add a project skill at .claude/skills/smoke-test/SKILL.md that lists your critical user flows and tells Claude to run them in a headless browser and report failures with screenshots and console errors. Type /smoke-test after a UI change, or name the skill verify (or simplify) so Claude Code runs it before each commit.

Who it's for & level

You already use Claude Code to write features and you're tired of it saying 'done' when the page is broken. You can run your app locally and know what your three to five critical user flows are. No testing framework experience needed.

Level
intermediate
Tools
Claude CodePlaywright
Roles
Developers Vibe coding
Goals
Ship an app
Tasks
Testing Code review
Time to set up
20–40 min (est.)
Cost
Playwright CLI is free and open source. Each run spends Claude Code usage from your plan or API budget; a 5-flow run is a few minutes of agent work (est., not measured).
Time saved
10–20 min per UI change vs clicking through by hand (est.)
Test status
Tested on 2026-10-15
Last verified
2026-10-15
Model versions
Claude Code v2.1.300, claude-sonnet-5-5
Tested by
Usecased tester (Claude Code v2.1.300 via API key) (agent)
Published
2026-10-15

Prerequisites

The problem

Coding agents edit UI code without ever seeing the page. They report success while a button no longer submits, a route 404s or the console fills with errors. Manually clicking through the same flows after every change is slow, so it gets skipped, and regressions ship.

Steps

  1. Install Playwright CLI. Run npm install -g @playwright/cli@latest and check it with playwright-cli --help. Playwright's own docs recommend the CLI plus skills over Playwright MCP for coding agents because it puts less page data into the model's context.
  2. Install the Playwright agent skill into the repo. In the app's root, run playwright-cli install --skills. It copies a skill to .claude/skills/playwright-cli/ that teaches Claude the command surface (snapshots, refs, sessions, screenshots, console, network). Commit it so teammates and cloud sessions get it.
  3. Write down your critical flows. Create .claude/skills/smoke-test/flows.md with 3–6 flows. For each one, give the start URL, the user actions in plain English, and one observable expected result (text on the page, a URL, a toast). Mark the ones that must always run as [critical].
  4. Create the smoke-test skill. Save the SKILL.md below to .claude/skills/smoke-test/SKILL.md. It injects the list of changed files with dynamic context, pre-approves only the playwright-cli and git diff commands it needs, and tells Claude to report, not fix.
  5. Run it. Start your dev server in another terminal, open claude in the repo and type /smoke-test http://localhost:3000. Claude opens a headless browser in a named session, runs each flow, and prints a pass/fail table with screenshots for failures.
  6. Triage and fix. Read the report, reproduce anything surprising yourself, then ask Claude to fix one failure at a time and re-run /smoke-test. Commit the skill and flows.md once they're stable.
  7. Optional: run it before every commit. Claude Code (v2.1.286+) tells Claude to run a project skill named verify or simplify right before each commit (docs and test-only changes are skipped). Rename the folder to .claude/skills/verify/ if you want that. A project skill at the repo root replaces the bundled /verify. Plugin skills and skills synced from claude.ai do not count. Turning off includeGitInstructions removes this instruction.

Exact prompt & config

1. Install Playwright CLI and its skill
npm install -g @playwright/cli@latest
playwright-cli --help
cd path/to/your-app
playwright-cli install --skills   # -> .claude/skills/playwright-cli/
2. .claude/skills/smoke-test/SKILL.md · Project skill. Invoke with /smoke-test [base-url].
---
name: smoke-test
description: Click through this app's critical user flows in a real headless browser with playwright-cli and report what broke, with screenshots and console errors. Use after UI changes, before a commit or PR, or when the user asks whether the app still works.
argument-hint: "[base-url]"
allowed-tools: Bash(playwright-cli *) Bash(git diff *) Read Grep Glob
---

## Files changed since the last commit
!`git diff --name-only HEAD`

## Task
Base URL: $ARGUMENTS (use http://localhost:3000 if empty).

1. Run `playwright-cli -s=smoke open <base-url>`. `open` navigates and writes a snapshot file under `.playwright-cli/`; it does not print the snapshot inline. If the page doesn't load, say which dev command in package.json starts the app and stop. Do not start long-running servers yourself.
2. Read ${CLAUDE_SKILL_DIR}/flows.md. Run every flow marked [critical], plus any flow whose route or component appears in the changed files above.
3. For each flow: take a `playwright-cli -s=smoke snapshot` to print the page snapshot with element refs, then use click / fill / press / select with those refs. Never guess a ref; snapshot again after navigation.
4. Check the flow's expected result with `playwright-cli -s=smoke find "<expected text>"` or the page URL. A flow passes only if the expected result is observed.
5. After each flow, run `playwright-cli -s=smoke console error` and `playwright-cli -s=smoke requests`. Record console errors and any 4xx/5xx responses even if the flow passed.
6. On failure, save `playwright-cli -s=smoke screenshot --filename=.smoke/<flow-name>.png`, then grep the codebase for the failing component or route and name the most likely file.
7. Finish with `playwright-cli -s=smoke close`.

## Report format
A markdown table: Flow | Result (PASS/FAIL) | Evidence | Console/network errors | Likely file.
Then a short "What to fix first" list. Do not edit application code in this skill. Only report.
3. .claude/skills/smoke-test/flows.md (example)
# Critical flows (edit for your app)

## sign-up [critical]
Start: /signup
Fill email "smoke+{timestamp}@example.com", password "Sm0ke-test-pass!", submit.
Expect: URL ends with /onboarding and the text "Welcome" is visible.

## log-in [critical]
Start: /login
Use the test account smoke@example.com / value of TEST_PASSWORD in .env.local.
Expect: text "Dashboard" is visible.

## create-project [critical]
Start: /dashboard
Click "New project", name it "Smoke test", save.
Expect: a toast "Project created" and the project appears in the list.

## pricing-page
Start: /pricing
Expect: three plan cards and a "Start free" button. No console errors.
4. Alternative: Playwright MCP instead of the CLI · Same idea over MCP. Heavier on context; Playwright now points coding agents to the CLI + skills.
claude mcp add playwright npx @playwright/mcp@latest
# then in SKILL.md, replace playwright-cli commands with:
# "use the playwright MCP tools (browser_navigate, browser_snapshot, browser_click, browser_console_messages)"

Sample output

Captured from our test run.

(Captured from our test run on 2026-10-15, Claude Code v2.1.300, Sonnet, against a tiny local test app. Lightly trimmed.)

Smoke test against http://127.0.0.1:3000: home passes, create-project fails.

| Flow                      | Result | Evidence                                                              | Console/network errors                                              | Likely file    |
|---------------------------|--------|-----------------------------------------------------------------------|---------------------------------------------------------------------|----------------|
| home [critical]           | PASS   | "Acme projects" heading present                                       | 404 on /favicon.ico (harmless)                                      | public/        |
| create-project [critical] | FAIL   | Clicked "New project"; no "Project created" status, list stayed empty | POST /api/projects -> net::ERR_EMPTY_RESPONSE; TypeError: Failed to fetch | server.py:13 |

Screenshot: .smoke/create-project.png

What to fix first
1. server.py:13: the POST handler reads body['title'], but dashboard.html sends {name: ...}. The KeyError drops the connection with no response. Read body['name'] instead.
No code was edited.

Why a skill and not just a prompt

You could paste “open the app and click around” into every session. A skill fixes three problems with that. Claude loads the instructions only when they’re needed, so they cost almost no context the rest of the time. The flow list lives in the repo next to the code, so it gets reviewed and updated like code. And allowed-tools pre-approves the playwright-cli commands for that turn, so the run doesn’t stop for a permission prompt on every click.

How the pieces fit

Making it stricter over time

Once a flow passes reliably, ask Claude: “Convert the create-project flow from flows.md into a Playwright test in tests/e2e/ using role-based locators, and run it with npx playwright test.” Then the check runs in CI without any agent at all, and the skill stays for exploratory checks after each change.

If you want the browser visible while it runs, start the session yourself with playwright-cli -s=smoke open http://localhost:3000 --headed, or run playwright-cli show to watch every active session in a dashboard. Headless sessions shut down after an hour idle by default (open --idle-timeout=<ms> changes that).

How it went

Tested with Claude Code v2.1.300 via API key (claude -p, Sonnet), on a three-page local test app with a planted bug, @playwright/cli 0.1.22. The first /smoke-test run took about 30 s and 8 turns ($0.05). It passed home, failed create-project with the right root cause (server.py read body['title'] while the client sent name), saved a screenshot and edited no code. It skipped the non-critical pricing flow, as SKILL.md says, because no changed file touched it. The fix-and-rerun loop worked: one minimal edit ($0.07), then a fresh /smoke-test passed both critical flows ($0.04, 22 s). Claude would not restart the running server itself and asked us to. With the folder renamed to verify, asking Claude to commit made it invoke the verify skill first, run the flows, and then commit (Sonnet, $0.04). Two surprises are listed under limitations: in non-interactive runs the model-invoked verify skill needs Skill(verify) permission, and the allowed-tools grant applied only when /smoke-test was the leading slash command. Total for all UC1 runs was about $0.34. Not tested: interactive sessions, a real Next.js/Vite app, logged-in flows.

Worth knowing

Also reported by

Other public write-ups of the same workflow. External links; we don't control their content.

FAQ

What are Claude Code skills?

A skill is a folder with a SKILL.md file: YAML frontmatter (name, description, optional fields like allowed-tools, context or arguments) plus instructions. Claude loads it when your request matches the description, or you run it directly with /skill-name. Project skills live in .claude/skills/ and personal ones in ~/.claude/skills/. Skills follow the open Agent Skills standard.

What's the difference between a skill and a slash command in Claude Code?

Custom commands were merged into skills. A file at .claude/commands/deploy.md and a skill at .claude/skills/deploy/SKILL.md both create /deploy. Skills add a folder for supporting files (like flows.md here), frontmatter that controls who can invoke them, and automatic loading when relevant.

Should I use Playwright MCP or Playwright CLI with Claude Code?

For coding agents, Playwright's own README recommends the CLI with its skill, because CLI calls avoid loading large tool schemas and accessibility trees into the context. Playwright MCP still makes sense for long exploratory sessions where keeping a live browser state matters more than token cost.

Can Claude Code test a web app automatically before each commit?

Yes. On Claude Code v2.1.286 or later, if a local project skill named verify or simplify exists and Claude is allowed to invoke it, Claude's commit instructions tell it to run that skill right before each commit (docs and test-only changes are skipped). Plugin and claude.ai-synced skills do not count; includeGitInstructions must stay on.

How do I share a Claude Code skill with my team?

Commit the .claude/skills/<name>/ folder to the repository. Everyone who opens Claude Code in that repo gets it, and cloud sessions load project skills from the cloned repo too. For several repos, package it as a plugin.

Related use cases