> workflow / testing
Testing
Prove behavior with automated tests and eval gates before promoting a release candidate — especially for AI-touched paths.
What should I do next?
Complete the steps and evidence below, clear the human gates, then continue to Security.
When to use
- Implementation PRs are merging or batched for an RC
- Risk tier demands eval gates, not only unit tests
- Flaky or snapshot-only coverage is blocking trust
- You need req → test traceability before security sign-off
Step-by-step
- Map acceptance criteria and plan slices to test cases; close gaps explicitly.
- Generate missing tests with test-generation.prompt; humans edit for realism.
- Add evals for LLM/agent paths; wire them as CI gates, not dashboards-only.
- Run regression suites; quarantine flakes with owners — do not ignore.
- For UI/flows, use Playwright (or equivalent) against critical journeys.
- Record coverage notes and known residual risk for the RC.
- Fail the stage if eval gates or critical path tests are red.
Decisions to make
- What is blocking vs advisory coverage for this risk tier
- Golden-set size and ownership for LLM evals
- When to add contract tests vs end-to-end only
- Ship with known residual risk vs hold the RC
Evidence to collect
- Test suite delta linked to plan items
- Eval results with gate status
- Flake list with owners or zero open flakes on critical paths
- Traceability notes: requirement → test/eval
Role-specific next actions
- PM: Confirm critical user journeys are covered, not just code paths.
- Architect: Require contract tests on new boundaries before RC.
- Engineer: Own unit/integration tests co-located with code.
- QA: Own risk-based suite, eval gates, and flake policy.
Inputs
- Requirements / plan acceptance criteria
- Merged implementation
- Risk tier and prior eval baselines
Outputs
- Test suite delta
- Eval gate results
- Coverage and residual-risk notes
Required prompts
- Test generation
Produces candidate tests from acceptance criteria and code.
Required skills
- Testing
Primary testing discipline and anti-patterns.
- Evaluation
Eval design and gate wiring for AI features.
Required MCP servers
- Playwright MCP
Browser journey automation for critical UI paths.
- Chrome DevTools MCP
Optional deep debugging of client-side failures.
Tooling & handbook
- Evals as quality gates
Make evals blocking in CI.
- Evaluation frameworks
Pick an eval approach that fits the workload.
- LLM-as-Judge
Use carefully with human baselines.
- Definition of Done
Includes test/eval evidence.
- Testing agent
Assists suite expansion under QA ownership.
Recommended agents & patterns
- Testing agent
Primary agent assist for this stage.
- Definition of Done
Quality bar agents and humans share.
Human approval gates
- Eval gate failure blocks release candidate promotion
- QA sign-off on residual risk for elevated tiers
Automation opportunities
- CI workflows for unit, integration, and eval suites
- Playwright critical-path smoke on PR and nightly
- Regression suites for prompts and agent tools
Enterprise best practices
- Golden sets for high-risk flows with change control
- Traceability from requirement → test/eval for audits
- Separate eval data from production PII
Common mistakes
- Snapshot-only tests for LLM features
- Skipping flake investigation
- Treating green unit tests as sufficient for agent tool changes