behavior-validator

Behavior Validator

Validate behavior from the outside. Code structure, agent claims, logs, and success messages prove nothing until the surface a real user depends on shows the result. "The code looks right" is not verification.

The source-blind rule

Validate without reading the implementation first. Use the public surface: UI, CLI output, API response, generated file, inbox, database row, calendar event, published page. Read the implementation only after external validation fails, or when you cannot design a safe test without it. Reading the code first biases you toward testing what the code does instead of what was promised.

Workflow

1. State the promised behavior in one sentence. If you cannot, you do not know what you are validating.
2. Pick the observable surface: browser page or screenshot, CLI stdout/stderr and exit code, API status and body, file content and metadata, email/calendar/CRM record, database row or log line, public URL.
3. Design the smallest test that proves the behavior. One action, one observable result.
4. Execute from a clean or known state. Stale state produces false passes.
5. Capture evidence: exact command and output, screenshot path, response body, record ID, timestamp, or log excerpt. Evidence must be reproducible and tied to the actual output.
6. Record pass, fail, or partial, and say explicitly what remains unproven.
7. On failure, hand off to systematic debugging or the relevant implementation skill. Do not patch blind from the validation seat.

Failure modes to hunt

• UI renders but action buttons do nothing
• API returns HTML error pages instead of JSON
• Automation reports success without creating or updating the target record
• File exists but content is stale or empty
• Works locally, broken through the public URL
• Outbound message drafted but never sent, or sent without logging
• Schedule exists with wrong timezone or recurrence

Good vs bad

Bad: "Verified: the submit handler calls the API and the API inserts the row. Code reviewed, looks correct." No external surface was touched. This is inspection, not validation.

Good: "Submitted the form as a fresh user at <URL>. API returned 201 (body captured). Row id=<ID> present in the table with the submitted values, created_at within the test window. Confirmation email arrived at the test inbox. Pass. Unproven: behavior under duplicate submission."

Verification

For each promised behavior, run the smallest external test and capture one artifact. Expect: the observable surface shows the promised result, and your evidence would let a stranger reproduce the check. If the surface cannot be reached (no render environment, no test account), record "unverified: <reason>" instead of pass. Never report verified from static code inspection alone.

Completion checklist

• [ ] Promised behavior written in one sentence
• [ ] Test executed against an external surface, not the source
• [ ] Clean or known starting state used
• [ ] Evidence captured: command, output, ID, path, or screenshot
• [ ] Result recorded as pass / fail / partial
• [ ] Unproven areas listed explicitly

Any box unchecked: not done. Fix or say so.

Red flags

Stop if you catch yourself saying any of these:

• "The code clearly does this, no need to run it"
• "The logs say success, that's enough"
• "It worked yesterday, same thing"
• "Too hard to test externally, I'll just read the implementation"
• "The framework handles that case"

Each one substitutes inference for observation. Close the loophole: if no external surface was observed, the honest status is "unverified", and you report it as such.

Footguns

• Trusting success messages. "Sync complete" from the tool is a claim, not evidence. Fix: check the target record, file, or page the sync was supposed to change.
• Validating from dirty state. A pre-existing row or cached page makes a broken flow look like it passed. Fix: create fresh test data or clear state, and check timestamps against the test window.
• Partial pass inflation. Happy path works, so the whole feature is declared done. Fix: list untested states (error, empty, duplicate, permission-denied) as unproven in the report.