Feature Loop

---
description: >-
  Plan-driven development loop for QGRE engine. Reads plan docs, compares against
  what's built, checks API correctness, finds gaps, builds the next piece, runs
  tests, and repeats until the plan is fully implemented with zero drift.
argument-hint: "[step name — e.g. 'step-0a', 'step-0d', 'step-1', 'all']"
allowed-tools: Read, Grep, Glob, Bash, Agent, Edit, Write, WebFetch, WebSearch
---

# Feature Loop — Plan-Driven Iterative Development (QGRE Engine)

You are a plan-driven development loop. Your job is to implement the QGRE engine
by iterating through a cycle until there are zero gaps between the plan
and the code. You do NOT invent. You do NOT improvise. You execute the plan.

## ABSOLUTE RULE: ALWAYS SEARCH EXA FIRST

This applies to EVERYTHING — not just bugs:
- Bug or crash → search Exa FIRST
- OOM or memory issue → search Exa FIRST
- API uncertainty → search Exa FIRST
- Architecture decision → search Exa FIRST
- Library compatibility question → search Exa FIRST
- Performance optimization approach → search Exa FIRST
- ANY technical uncertainty → search Exa FIRST

NEVER trust training data for: Unsloth APIs, vLLM behavior, PyTorch internals,
Triton kernel patterns, bitsandbytes usage, Liger Kernel integration, CUDA memory
management, or any library that changes faster than monthly.

The cost of one Exa search: 3 seconds.
The cost of a wrong assumption: hours of debugging or silent training corruption.

If you catch yourself writing code without having searched Exa for the specific
technique or API you're using, STOP and search first. No exceptions.

You also have `/tech-scan` — use it for broader research on training techniques,
GRPO best practices, memory management strategies, and library compatibility.
Run it whenever you're facing a systemic issue (not just a single bug).

REMEMBER: We are building a CUSTOM ENGINE. We have FULL CONTROL over every line.
If an existing library function doesn't fit, we don't hack around it — we write
our own implementation. We have the source code for Unsloth, vLLM, NeMo RL, and
PyTorch. Any bug, any limitation, any OOM is fixable because we own the pipeline.

## Input

The user provides a build step name (e.g. `step-0d`, `step-1`) or `all`.

Plan docs live at:
- `docs/PLAN-engine-improvements.md` — ACTIVE improvements plan (Items 0-10, CPA findings, research references)
- `docs/TECH-SCAN-2026-03-19.md` — live research findings per technology
- `docs/PLAN.md` — original engine build plan (COMPLETE — reference only)
- `docs/PILLARS.md` — six pillars decomposition (reference only)
- `docs/SPECIAL-TOKENS-SUPERPOWER.md` — VPRM spec (reference only)

The ACTIVE plan is `docs/PLAN-engine-improvements.md`. Read it FIRST every loop iteration.
If an item's spec is not in the improvements plan, STOP and tell the user.

## Project Structure

qgre/ init.py — package root, exports RewardResult types.py — RewardResult dataclass config.py — engine config dataclass (Step 1) segments.py — segment_completion() + STEP_QUALITIES (Step 0d) advantages.py — QGREStepAdvantageEstimator (Step 0d) data.py — DataLoader: parquet → tokenize → pad → batch (Step 0e) checkpoint.py — GameState serializer + checkpoint resume (Steps 0a, 0f) logging.py — MLflow tracking + JSONL dump (Steps 4, 6) trainer.py — QGRETrainer (Step 1) generation.py — UnslothBackend: vLLM colocated generation (Step 2) lora_verify.py — LoRA weight sync verification (Step 0g) fused_logprobs.py — Chunked logprobs without full logits materialization triton_logprobs.py — Triton fused lm_head→logprobs kernel (BLOCK_V=128) nemo_extracted/ init.py — NeMo RL attribution loss_functions.py — ClippedPGLossFn (Step 0b) kl.py — KL calculation + kl_cov (Step 0b) logits.py — Log prob computation + selective_log_softmax (Step 0b) llds.py — LLDS loss (arXiv:2512.04220) LICENSE — Apache-2.0 attribution examples/ hypergraph/ config.yaml — full QGRE config (model, data, generation, algorithm, training, logging) reward_fn.py — reward function returning RewardResult with per-quality scores math/ config.yaml — minimal SPO config reward_fn.py — math reward function tests/ conftest.py — fixtures (synthetic batches, known token IDs, mock models) test_checkpoint.py — Steps 0a, 0f, 5 test_nemo_extracted.py — Step 0b test_segments.py — Step 0d (segmentation) test_advantages.py — Steps 0c, 0d, 8 (advantages + credit assignment) test_data.py — Step 0e test_trainer.py — Step 1 test_wiring.py — Steps 2-3 (GPU required) test_logging.py — Steps 4, 6 test_consistency.py — Step 7 (internal consistency, NOT verl equivalence) test_smoke.py — GPU smoke test test_triton_logprobs.py — Triton kernel tests

## Build Sequence (from PLAN.md)

Prerequisites: 0a: GameState serializer (checkpoint.py) 0b: NeMo RL extraction (nemo_extracted/*.py) 0c: Batch reward tensor construction (advantages.py — partial) 0d: QGREStepAdvantageEstimator (segments.py + advantages.py) — CORE ALGORITHM 0e: DataLoader (data.py) 0f: Checkpoint resume (checkpoint.py) 0g: LoRA verification harness (new module)

Assembly: 1: QGRETrainer (trainer.py + config.py) 2: Wire Unsloth + vLLM generation 3: Wire reward function 4: Wire MLflow tracking 5: Wire checkpoint save/resume 6: Wire JSONL dump 7: Equivalence test 8: Credit assignment test

## The Loop

Execute this cycle. Each iteration does ALL 6 phases. After phase 6,
if gaps remain, start phase 1 again. Continue until phase 6 finds zero gaps.

PHASE 1: READ THE PLAN ↓ PHASE 2: READ WHAT'S BUILT ↓ PHASE 3: CHECK API CORRECTNESS ↓ PHASE 4: FIND GAPS (plan vs built) ↓ PHASE 5: BUILD THE NEXT GAP ↓ PHASE 6: VERIFY ```