Qa

# /qa — Test, Fix, Verify

> Read CLAUDE.md before proceeding so you have current project context.

Run the test suite, diagnose failures, fix them, and verify. Repeat until clean.
**Never report done until tests are actually passing** — show the output.

---

## Usage

/qa [quick|standard|exhaustive] [--diff-only]

- `quick` — run fast unit tests only (`tests/unit/`)
- `standard` — run all tests (`tests/`) — **default**
- `exhaustive` — run all tests + type checks + lint
- `--diff-only` — only run tests for files touched in recent `git diff`

---

## Step 0 — Check Prerequisites

```bash
python -m pytest --version 2>/dev/null || echo "pytest not found"

If pytest is missing:

"pytest not installed. Run: pip install pytest or add it to your dev dependencies."

Check test directory exists:

ls tests/ 2>/dev/null || echo "No tests/ directory found"

---

Step 1 — Determine Scope

If --diff-only:

git diff --name-only HEAD 2>/dev/null

Map each changed file to its test counterpart:

• src/ysp/rules/dep_confusion.py → tests/unit/rules/test_dep_confusion.py
• src/ysp/transport/gh_adapter.py → tests/unit/transport/test_gh_adapter.py

Run only those test files. If no test files found for changed source, fall back to standard.

If quick: python -m pytest tests/unit/ -v If standard: python -m pytest tests/ -v If exhaustive: run standard + type checks + lint (see Step 4).

---

Step 2 — Run Tests

python -m pytest <scope> -v --tb=short 2>&1

Record:

• Total: X passed, Y failed, Z errors
• Which tests failed (file:line:test_name)
• The failure output (assertion error or exception traceback)

---

Step 3 — Fix Failures (One at a Time)

For each failing test:

1. Read the test — understand what it's asserting.
2. Read the source — find the code the test exercises.
3. Identify the gap — is the source wrong, or is the test wrong?
   • If the source is wrong: fix it (apply /develop-feature discipline — backward compat, logging).
   • If the test is wrong (tests a removed/changed behavior): update the test.
   • If both need updating: fix source first, then test.
4. Never delete a test to make it pass unless the behavior it tests was intentionally removed.

After fixing each failure, re-run just that test to confirm:

python -m pytest <test_file>::<test_name> -v

---

Step 4 — Exhaustive Mode (additional checks)

Run type checks:

python -m mypy src/ --ignore-missing-imports 2>&1 | tail -20

Run linter:

python -m ruff check src/ 2>&1 | tail -20

Fix any errors found before proceeding.

---

Step 5 — Final Verification Pass

After all fixes, run the full scope once more:

python -m pytest <scope> -v 2>&1 | tail -30

Only declare done when this shows X passed, 0 failed.

---

Step 6 — Report

QA REPORT
=========
Scope    : <quick|standard|exhaustive> [--diff-only]
Result   : PASS (N tests) / FAIL (N failures remaining)
Fixed    : <list of test_name: what was fixed>
Skipped  : <list with reason>
Output   :
  <last 15 lines of pytest output>

---

ysp-specific test conventions

From docs/scope/detailed_design.md:

• Rule tests use should_trigger/ and should_not_trigger/ fixture subdirectories.
• Each rule fixture is a minimal file that either does or does not match the rule.
• Transport tests must not make real network calls — use unittest.mock.patch.
• Log ID tests: assert that OP- and ERR- IDs appear in log output for the happy and error paths.
• Exception swallowing: grep -r "except.*pass" src/ should return nothing — run this as part of exhaustive QA.

grep -rn "except.*:$" src/ | grep -v "#" | grep -v "test_"

Flag any except: blocks with empty bodies as defects. ```