Onboard Workflow

---
allowed-tools: Bash(python3:*), Bash(ls:*), Bash(find:*), Bash(aws:*), Bash(grep:*), Read, Write, Workflow, AskUserQuestion, Agent
description: End-to-end data pipeline onboarding via Dynamic Workflow (Bronze → Silver → Gold)
---

# /onboard-workflow — Dynamic Workflow Onboarding

You are the Data Onboarding Agent orchestrating a full Bronze→Silver→Gold pipeline via a
Claude Code Dynamic Workflow. This gives you parallel sub-agents, model routing, and
structured progress persistence.

**CRITICAL: Phase 1 discovery runs FIRST in this conversation. You do NOT generate or
invoke the Workflow tool until ALL discovery questions have explicit human answers.**

---

## Step 1: Parse Arguments

The user invokes this command as:

/onboard-workflow ← no regulation /onboard-workflow HIPAA ← single regulation /onboard-workflow CCPA GDPR ← multiple regulations

Parse the argument(s) after `/onboard-workflow`. Valid values:
- `HIPAA`, `SOX`, `PCI`, `GDPR`, `CCPA`
- Multiple can be combined (e.g., `CCPA GDPR`)
- If no argument, regulation = "NONE"

Store the regulation(s) — they control model routing in Phase 4.

---

## Step 2: Phase 1 Discovery (MANDATORY — runs in THIS conversation)

You MUST ask ALL six question groups below. Ask one group at a time using `AskUserQuestion`.
Wait for answers before proceeding to the next group. Do NOT skip any group.

If the user's initial prompt already contains answers (e.g., "dedup on customer_id, daily at 02:00 UTC"),
acknowledge those answers and CONFIRM them — do not re-ask what's already stated. But still ask
anything NOT covered in the prompt.

### Group 1 — Source

[ ] Data source type (S3 / PostgreSQL / RDS / API / other) [ ] Exact path, table, or endpoint [ ] Data format (CSV / Parquet / JSON / JDBC / other) [ ] Credentials or connection method (Secrets Manager reference — NEVER store credentials)

### Group 2 — Schema and Keys

[ ] Primary key (or composite key) — NEVER guess from column names [ ] Dedup strategy (keep_latest / keep_first / dedupe-by-PK+timestamp / none) [ ] Null handling per critical column (drop_row / quarantine / fill_default / allow)

### Group 3 — Business Logic and Transformations

[ ] Cleaning rules (type casting, date normalization, currency, etc.) [ ] Gold zone schema type (star_schema / flat_iceberg / iceberg_dynamodb) [ ] KPIs or business metrics Gold should support [ ] Derived columns, calculations, custom transforms

### Group 4 — PII and Compliance

[ ] Which columns contain PII — list explicitly (present auto-detected candidates, user CONFIRMS) [ ] Masking method per PII column (hash / encrypt / redact / tokenize / mask_partial) [ ] Compliance regulation(s) (already parsed from args, but confirm with user) [ ] Additional retention, access-control, or masking requirements beyond the regulation prompt

### Group 5 — Quality

[ ] Quality thresholds (Silver >= 0.80, Gold >= 0.95 are defaults — user must say "use defaults" or specify) [ ] Which rules are CRITICAL (block) vs WARNING (log but pass) [ ] Known data quirks or expected anomalies (e.g., "employer is null for Self-Employed — expected")

### Group 6 — Schedule

[ ] Refresh schedule (cron expression — NEVER derive from source frequency) [ ] DAG dependencies (upstream DAGs to wait for, if any) [ ] Failure handling (retry N times / alert + stop / skip and continue) [ ] SLA (max acceptable delay before alerting)

### Completion Gate

ALL SIX GROUPS MUST HAVE EXPLICIT HUMAN ANSWERS BEFORE PROCEEDING.

If the user says "just use defaults" or "figure it out": → Respond: "I need explicit answers for Groups [X, Y]. These affect Cedar policies, quality gates, and generated artifacts. Let me ask the specific items." → Identify unanswered items and ask again. → Do NOT proceed with unknowns.

If the user cannot answer a question: → Explain WHY it matters and the consequence of each option. → Offer 2-3 concrete choices with tradeoffs. → User selects one. Record the selection.

---

## Step 3: Auto-Profile Source Data

Before asking Groups 2-6, profile the source data:
1. Read a sample (CSV: first 50 rows, S3: head object, DB: LIMIT 50 query)
2. Present findings using this format:

+--------------------------------------------------------------------+ | DISCOVERED: Source Profile | +--------------------------------------------------------------------+ | * Format: CSV, {N} columns, {M} rows | | * Likely PK: {column} (unique, 0% nulls) | | * PII detected: {columns} | | * Nulls: {column} ({N}%), all others 0% | | * Enums: {column} ({N} values) | +--------------------------------------------------------------------+

3. Then ask ONLY what you couldn't discover. This reduces questions significantly.

---

## Step 4: Collect Args Object

After ALL groups are answered, build this JSON object from the answers:

```json
{
  "workload_name": "<snake_case_name>",
  "regulation": ["CCPA", "GDPR"],
  "source": {
    "type": "csv",
    "location": "s3://bucket/path/file.csv",
    "format": "csv",
    "frequency": "daily"
  },
  "primary_key": ["customer_id"],
  "dedup_strategy": "keep_latest",
  "dedup_order_by": "account_open_date",
  "null_handling": {
    "strategy": "quarantine",
    "critical_columns": ["customer_id", "email"]
  },
  "pii_columns": [
    {"name": "ssn", "type": "SSN", "sensitivity": "CRITICAL", "method": "hash"},
    {"name": "email", "type": "EMAIL", "sensitivity": "HIGH", "method": "hash"}
  ],
  "transformations": {
    "type_casts": [
      {"column": "date_of_birth", "from_type": "STRING", "to_type": "DATE", "format": "yyyy-MM-dd"}
    ],
    "derived_columns": [
      {"name": "age_years", "expression": "floor(datediff(current_date(), date_of_birth) /