The full product evaluation loop

A comprehensive product-quality workflow that evaluates realistic scenarios across every major capability, fixes weak outcomes, a…

Matthew Berman 2.2k updated 1mo ago
Claude CodeGeneric
View source ↗

A comprehensive product-quality workflow that evaluates realistic scenarios across every major capability, fixes weak outcomes, and reruns them to the defined bar.

Use when

Use this for an exhaustive, end-to-end application QA pass when a production-like local environment and complete interactive-surface coverage matter more than a narrow regression or sample of major features.

The loop

Build sanitized, production-scale local data under production-like settings. Inventory every user-facing feature, role, route, button, input, modal, state, and workflow; define documented acceptance criteria and finite risk-based edge cases for each. Test as a real user, logging every bug with reproduction evidence. Review findings for shared causes and dependencies; implement coherent fixes with regression tests, then rerun the full inventory. Stop at a clean pass or blocked handoff. Ask before production, sensitive data, or destructive actions.

Steps

  • Build a sanitized or synthetic production-scale local dataset, mirror safe production settings, and record unavoidable differences.
  • Inventory every user-facing feature, role, route, control, state, and workflow; define documented acceptance criteria and a finite risk-based edge-case set for each item.
  • Exercise every inventory item as a real user under its normal and defined edge-case conditions, logging each bug immediately with reproducible evidence.
  • Review the complete bug set for shared causes, dependencies, and conflicting fixes, then implement the smallest coherent solution with regression coverage.
  • Rerun affected paths and the complete inventory; stop only at a clean full pass or an explicit blocked handoff.

Verification

Every inventoried product surface meets its documented acceptance criteria. The final full regression run covers every inventoried surface and its finite risk-based edge cases in the production-like local environment, with each reproducible bug fixed and backed by evidence.

Why it works

A finite surface inventory prevents major controls and states from disappearing behind a few happy-path scenarios. Reviewing all findings before fixing them exposes shared causes and interactions, while the final full run catches changes that repair one path but weaken another.

Maintain The full product evaluation loop?

Let people know it's listed here — add the badge (live metrics, light/dark aware) or a plain link to your README or docs.

The full product evaluation loop on getagentictools
[![The full product evaluation loop on getagentictools](https://getagentictools.com/badge/loops/full-product-evaluation-loop.svg)](https://getagentictools.com/loops/full-product-evaluation-loop?ref=badge)