Run Test Plan
We are executing the test plan. All implementation is complete. Now we verify it works.
We are executing the test plan. All implementation is complete. Now we verify it works.
## Reference Documents
- **Test Plan:** `notes/test_plan.md`
- **Implementation Plan:** `notes/impl_plan.md`
- **Specs:** `notes/specs.md`
- **Test Status JSON:** `notes/test-status.json`
- **Test Results Log:** `notes/test-results.md`
---
## Phase 1: Initialize
### Step 1: Check for Existing Run (Resumption)
Before creating anything, check if a previous test run exists:
1. Check if `notes/test-status.json` exists
2. Check if there are existing tasks via `TaskList`
**If both exist and tasks have results:**
- This is a **resumed run** — skip to Phase 2 (Step 7)
- Announce: "Resuming previous test run. Skipping already-passed tests."
- Only execute tasks that are still `pending` or `fail` (with fixAttempts < 3)
**If no previous run exists (or files are missing):**
- Continue with fresh initialization below
### Step 2: Read the Test Plan
Read `notes/test_plan.md` and extract ALL test cases. Auto-detect the TC-ID pattern used (e.g., `TC-001`, `TC-101`, `TC-5A`, etc.).
For each test case, note:
- TC ID
- Name
- Priority (Critical / High / Medium / Low — default to Medium if not stated)
- Preconditions
- Test steps and expected outcomes
- Test data (if any)
- Dependencies on other test cases (if any)
### Step 3: Analyze Test Dependencies
Determine which test cases depend on others. Common dependency patterns:
- A "saves data" test may depend on a "displays default" test
- A "form submission" test may depend on "form validation" tests
- An "end-to-end" test may depend on individual component tests
If no clear dependencies exist between test cases, treat them all as independent.
### Step 4: Create Tasks
Use `TaskCreate` to create one task per test case. Set `blocked_by` based on the dependency analysis.
**Task description format:**
Test [TC-ID]: [Test Name] Priority: [Priority]
Preconditions:
- [Required state before test]
Steps:
| Step | Action | Expected Result |
|---|---|---|
| 1 | [Action] | [Result] |
| 2 | [Action] | [Result] |
Test Data:
Expected Outcome: [Final verification]
fixAttempts: 0 result: pending lastTestedAt: null notes:
### Step 5: Generate Test Status JSON
Create `notes/test-status.json`:
```json
{
"metadata": {
"testPlanSource": "notes/test_plan.md",
"totalIterations": 0,
"maxIterations": 50,
"startedAt": null,
"lastUpdatedAt": null,
"summary": {
"total": "<count>",
"pending": "<count>",
"pass": 0,
"fail": 0,
"knownIssue": 0
}
},
"testCases": {
"<TC-ID>": {
"name": "Test case name",
"priority": "Critical|High|Medium|Low",
"status": "pending",
"fixAttempts": 0,
"notes": "",
"lastTestedAt": null
}
},
"knownIssues": []
}
Step 6: Initialize Test Results Log
Create notes/test-results.md:
# Test Results
**Test Plan:** notes/test_plan.md
**Started:** [CURRENT_TIMESTAMP]
## Execution Log
Verify Initialization
Use TaskList to confirm:
- All TC-IDs from the test plan have a corresponding task
- Dependencies are correctly set via
blocked_by - All tasks show
result: pending
Cross-check task count matches summary.total in notes/test-status.json.
Phase 2: Execute Tests
Step 7: Determine Execution Order
Use TaskList to read all tasks and their blocked_by fields. Determine sequential execution order:
- Tasks with no
blocked_by(or all dependencies resolved) come first - Tasks whose dependencies are resolved come next
- Continue until all tasks are ordered
For resumed runs: Skip tasks where result is already pass or known_issue.
Step 8: Execute One Task at a Time
For the next eligible task, spawn ONE sub-agent with the instructions below.
One sub-agent at a time. Do NOT spawn multiple sub-agents in parallel.
Sub-Agent Instructions
You are a test execution sub-agent. You have ONE job: execute and verify ONE test case.
Read your task using
TaskGetto get the full descriptionParse the test steps from the description (everything above the
---separator)Parse the metadata from below the
---separatorRead CLAUDE.md for environment details, URLs, and credentials
Execute the test:
Using browser automation:
- Navigate to URLs specified in the test steps
- Click buttons/links as described
- Fill form inputs with the test data provided
- Take screenshots at key verification points
- Read console logs for errors
- Verify DOM state matches expected outcomes
Follow the test plan steps EXACTLY. Do not skip steps.
Determine the result:
PASS if:
- All expected outcomes verified
- No unexpected console errors
- UI state matches test plan
FAIL if:
- Any expected outcome not met
- Unexpected errors
- UI state doesn't match
If PASS: Update the task description metadata via
TaskUpdate:--- fixAttempts: 0 result: pass lastTestedAt: [CURRENT_TIMESTAMP] notes: [Brief description of what was verified]Mark the task as
completed.If FAIL and fixAttempts < 3:
a. Analyze the root cause b. Implement a fix in the codebase c. Increment fixAttempts and update via
TaskUpdate:--- fixAttempts: [previous + 1] result: fail lastTestedAt: [CURRENT_TIMESTAMP] notes: [What failed, root cause, what fix was applied]d. Re-run the test steps to verify the fix e. If now passing, set
result: passand mark task ascompletedf. If still failing and fixAttempts < 3, repeat from (a)If FAIL and fixAttempts >= 3: Mark as known issue via
TaskUpdate:--- fixAttem
Maintain Run Test Plan?
Let people know it's listed here — add the badge (live metrics, light/dark aware) or a plain link to your README or docs.
[Run Test Plan on getagentictools](https://getagentictools.com/loops/nathanonn-test-results?ref=badge)