Verify Prd

Verify that a PRD has been fully implemented after a Sandcastle Ralph loop. This catches the class of bugs where issues get close…

brandtam 2 updated 2mo ago
Claude CodeGeneric
View source ↗
# Verify PRD

Verify that a PRD has been fully implemented after a Sandcastle Ralph loop. This catches the class of bugs where issues get closed by commit messages on branches that were never merged, where user stories were covered by closed issues but the code doesn't actually exist, and where concurrent agents created conflicting database migrations.

## Process

### 1. Load the PRD

Ask the user for the PRD GitHub issue number. Fetch it with `gh issue view <number>`.

Identify:

- All numbered user stories in the PRD
- The feature branch name (ask the user if not obvious)

### 2. Find all child issues

Search for issues that reference the PRD:

```bash
gh issue list --search "<number> in:body" --state all --limit 100 --json number,title,state

Build a map of issue number → title → state.

3. Check for unmerged branches

This is the most critical check. Sandcastle creates sandcastle/issue-* branches for each issue. Verify every one was merged:

# Find sandcastle issue branches NOT merged into the feature branch
git branch --no-merged <feature-branch> | grep "sandcastle/issue-"

For each unmerged branch, check if it corresponds to a closed child issue — this is the orphan pattern where GitHub closed the issue (because the commit message said closes #N) but the code never reached the feature branch.

Cross-reference:

# What commits are on the unmerged branch that aren't on the feature branch?
git log --oneline <feature-branch>..<unmerged-branch>

Also check for orphaned session branches:

git branch --no-merged <feature-branch> | grep "sandcastle/claude-code"

For each, check what's on them that's not on the feature branch.

4. Check for Sandcastle side effects

Sandcastle may modify local git configuration during runs (e.g., embedding access tokens in remote URLs so Docker containers can push branches). Verify these haven't been left behind:

# Check if the git remote URL has credentials embedded
git remote -v

If the remote URL contains x-access-token: or any token/password, flag it:

⚠️ Git remote URL contains embedded credentials:
  origin  https://x-access-token:ghp_XXXX@github.com/org/repo.git

This was likely set by Sandcastle for container auth. Fix with:
  git remote set-url origin https://github.com/org/repo.git

Also check for any leftover Sandcastle worktree state:

# Check for stale worktrees
git worktree list

If there are worktrees beyond the main one, note them — they may be leftover from failed Sandcastle runs.

5. Audit database migration consistency

Sandcastle runs multiple agents concurrently in separate Docker containers. Each agent may generate Drizzle migrations independently. Since they all start from the same base state, they produce migrations with the same sequence number (e.g., two agents both generate 0003_*.sql). When merged, this creates conflicts in the Drizzle journal and snapshot metadata.

Run these checks for every migration directory in the project (check CLAUDE.md or drizzle.config.ts for locations — typically apps/bookbuilder/src/lib/server/migrations/ and packages/platform/src/db/migrations/):

4a. Journal integrity

# Read the journal
cat <migrations-dir>/meta/_journal.json

Verify:

  • Valid JSON — parse the journal with strict JSON.parse; trailing commas, missing brackets, etc. will break drizzle-kit generate. Fix any syntax errors immediately.
  • Sequential idx values — entries should be 0, 1, 2, 3... with no gaps or duplicates
  • Unique tags — no two entries share the same tag (filename)
  • Monotonic timestamps — each entry's when should be greater than the previous
  • Matching files — every tag in the journal has a corresponding .sql file and meta/<idx>_snapshot.json

4b. Migration file consistency

# List migration SQL files
ls <migrations-dir>/0*.sql

# List snapshot files
ls <migrations-dir>/meta/0*_snapshot.json

Verify:

  • 1:1 mapping — every .sql file has a journal entry and a snapshot; every journal entry has a .sql file and a snapshot
  • No orphan files — no .sql files or snapshots without journal entries (leftover from a merge conflict)
  • No duplicate sequence numbers — no two .sql files with the same NNNN_ prefix

4c. Schema-migration drift

The Drizzle schema files (TypeScript) are the source of truth. The latest snapshot should match what drizzle-kit generate would produce. Check for drift:

# Dry-run generate to see if there are pending changes
pnpm --filter <package> db:generate --dry-run 2>&1

If this reports changes, either:

  • The schema was modified after the last migration was generated (common after code review fixes)
  • A migration from one agent included schema changes that another agent's migration didn't account for

When drift is detected, note what the pending changes are and whether they're intentional (e.g., a NOT NULL constraint added during code review) or accidental (merge artifact).

4d. Cross-branch migration conflicts

Check whether unmerged branches (from step 3) contain their own migrations that would conflict:

# For each unmerged branch, check for migration files
git diff --name-only <feature-branch>...<unmerged-branch> | grep -E "migrations/[0-9]|_journal.json|snapshot"

If an unmerged branch has migrations at the same sequence number as migrations already on the feature branch, merging it will create a conflict. Document the conflict and recommend resolution (typically: merge the branch, then delete the conflicting migration and re-generate from the merged schema).

4e. Migration content review

For each migration file added in this PRD branch (not on main):

git diff --name-only main...<feature-branch> | grep "migrations/0.*\.sql"

Read each migration and verify:

  • No hand-edited generated files — Drizzle-generated migrations should not be manually modified. If th

Maintain Verify Prd?

Let people know it's listed here — add the badge (live metrics, light/dark aware) or a plain link to your README or docs.

[Verify Prd on getagentictools](https://getagentictools.com/loops/brandtam-verify-prd?ref=badge)