testing-strategy

by clawkk

Install this Skill

curl -fsSL -o testing-strategy.zip https://skill.seembox.com/api/v1/skills/testing-strategy/download && unzip testing-strategy.zip

Download testing-strategy.zip

SKILL.md About

---
name: testing-strategy
description: Deep testing strategy workflow—risk mapping, test pyramid, levels of isolation, flakiness, data, CI gates, and quality signals beyond coverage %. Use when designing test approach, fighting flaky CI, or restructuring QA vs dev ownership.
---

# Testing Strategy (Deep Workflow)

Testing strategy answers: **what failures would hurt users**, **what’s cheap to catch**, and **what signals we trust in CI**. Coverage percentage alone is a weak proxy—**risk alignment** matters.

## When to Offer This Workflow

**Trigger conditions:**

- New service or major refactor; “what should we test?”
- Flaky CI, long runtimes, or tests nobody trusts
- Debate: unit vs integration vs e2e; QA headcount vs automation

**Initial offer:**

Use **six stages**: (1) risk & quality goals, (2) pyramid & layers, (3) design per layer, (4) data & environments, (5) CI & gates, (6) observability of test health. Confirm **release cadence** and **regulatory** needs.

---

## Stage 1: Risk & Quality Goals

**Goal:** Connect tests to **user impact** and **business risk**.

### Questions

1. Worst **failure categories**: payments wrong, data leak, outage, wrong advice (AI)?
2. **SLO** for critical paths—what must never break silently?
3. **Change velocity**—how fast must PRs merge safely?

### Output

**Risk register** → **test priorities** (not every line equally important).

**Exit condition:** Top **5 risks** have explicit **test intent**.

---

## Stage 2: Pyramid & Layers

**Goal:** **Many fast tests**, **some integration**, **few e2e**—proportion tuned to risk.

### Layers (typical)

- **Unit**: pure logic, cheap, deterministic
- **Integration**: DB, queue, real dependencies in containers—slower but valuable
- **Contract**: between services—consumer-driven contracts when decoupled teams
- **E2E**: full stack—expensive; **minimal happy path + critical regressions**

### Anti-patterns

- **E2E-only** (slow, flaky)
- **Mock everything** (misses real integration bugs)

**Exit condition:** Written **policy**: what belongs in each layer for this codebase.

---

## Stage 3: Design Per Layer

**Goal:** Tests are **readable**, **stable**, and **debuggable**.

### Unit

- **Given/when/then** clarity; **avoid** testing implementation details
- **Property-based** tests for tricky invariants (dates, money, parsers)

### Integration

- **Testcontainers** or docker-compose in CI; **migrations** applied
- **Parallel** safe—unique DB schemas or transactions

### E2E

- **Stable selectors** (data-testid); **retry** policy disciplined—**fix flakes**, don’t hide them
- **Seed data** minimal; **idempotent** setup

**Exit condition:** **Flake** classification process exists (quarantine + ticket).

---

## Stage 4: Data & Environments

**Goal:** **Representative** data without **PII** leakage.

### Practices

- **Fixtures** versioned; **factories** for variations
- **Anonymized** prod-like datasets for perf tests—**governance** for access
- **Env parity**: staging behaves like prod enough for meaningful e2e

**Exit condition:** Data **generation** documented; **secrets** not in tests.

---

## Stage 5: CI & Gates

**Goal:** **Fast feedback** on PRs; **nightly** heavier suites if needed.

### Tiers

- **PR**: lint, unit, fast integration subset
- **Main**: full integration; **optional** e2e against ephemeral env
- **Release**: smoke + canary in prod

### Metrics

- **Flake rate**, **duration**, **quarantined** tests count—**visible**

**Exit condition:** **Merge policy** tied to green checks; **exceptions** process defined.

---

## Stage 6: Test Health & Culture

**Goal:** Tests are **owned** like features.

### Practices

- **Ownership** per suite; **on-call** for CI when org size supports
- **Delete** tests that don’t pay rent—**or** fix them

---

## Final Review Checklist

- [ ] Risks mapped to test layers
- [ ] Pyramid policy documented
- [ ] Flake management process exists
- [ ] CI tiers match team velocity
- [ ] Data/fixture strategy safe and maintainable

## Tips for Effective Guidance

- Recommend **testing seams**: boundaries where contracts are stable.
- Warn against **snapshot abuse** for large UI—**diff noise** kills trust.
- For **AI/LLM**, discuss **eval harnesses** beyond classic unit tests.

## Handling Deviations

- **Legacy untestable code**: **characterization tests** then refactor seams.
- **Startup speed**: **smoke** + **critical path** first; expand as pain appears.

_meta.json Metadata

{
  "ownerId": "kn76s36x99btbdct5mxefxwmth8310m2",
  "publishedAt": 1774368154849,
  "slug": "testing-strategy",
  "version": "1.0.0"
}