automation-audit-ops

Automation Audit Ops

Audit first, fix later. The deliverable is an evidence-backed inventory with one call per surface: keep, merge, cut, or fix next. Never claim an automation is live because a config references it. Configured is not working.

Ground rules

• Start read-only. No fixes unless the user explicitly asked for them.
• Classify every item by live state, in this exact ladder: configured, authenticated, recently verified, stale or broken, missing entirely.
• Do not merge or delete overlapping surfaces until the evidence table exists.
• If the state of something is ambiguous, say so. An honest gap beats a fake complete audit.

Workflow

1. Inventory the real surface

Read what actually exists before theorizing:

• scheduled jobs (cron, platform schedulers, hosted triggers)
• CI workflows and scheduled pipeline runs
• repo hooks and local hook scripts
• MCP configs and enabled servers
• connector-backed and app-backed integrations
• wrapper scripts and automation entrypoints

Group by surface: local runtime, repo CI, connected external systems, messaging and notifications, billing and customer ops, research and monitoring.

2. Classify live state and problem type

For each item, assign one live state from the ladder above, then one problem type: active breakage, auth outage, stale status, overlap or redundancy, or missing capability.

3. Trace the proof path

Back every claim with a concrete source: file path, workflow run URL, log line, config entry, recent command output, or exact failure signature. A claim with no proof path goes in an "unverified" column, not in the findings.

4. Deliver keep / merge / cut / fix-next

One call per overlapping or suspect surface. The value of the audit is collapsing noise into one canonical lane per job, not preserving every historical path. Order the fix-next list by signal value: name the broken high-signal path before low-value redundancy.

Output format

CURRENT SURFACE
- automation | source | live state | proof

FINDINGS
- active breakage | overlap | stale status | missing capability

RECOMMENDATION
- keep | merge | cut | fix next (ordered)

NEXT MOVE
- the exact lane to strengthen first

Good vs bad

Bad: "The nightly sync is live, it is defined in the scheduler config." Configuration is not evidence of execution.

Good: "Nightly sync: configured and authenticated, last successful run <TIMESTAMP> per run log line <REF>, but the downstream sheet was last modified 3 weeks ago. State: stale or broken. Call: fix next."

Verification

For each item marked "recently verified", confirm the proof is a run record or output artifact dated within the item's expected cadence. Expect every live claim to survive the question "show me the run". If any claim rests only on config or memory, downgrade it to configured and rerun the classification.

Completion checklist

• [ ] Every discovered automation appears in the inventory with a source
• [ ] Every item has exactly one live-state label
• [ ] Every important claim cites a proof path
• [ ] Ambiguous states flagged as ambiguous, not guessed
• [ ] Each overlapping surface has one keep/merge/cut/fix-next call
• [ ] Fix-next list ordered by signal value
• [ ] No fixes performed unless the user asked

Any box unchecked: not done. Fix or say so.

Footguns

• Answering from memory. The live inventory can be read; reading it beats recalling it every time. Fix: enumerate configs, schedules, and logs before writing a single finding.
• Treating "present in config" as "working". The most common false positive in automation audits. Fix: demand a recent run record or output artifact for any "live" label.
• Scope creep into a rewrite. The user asked what exists, and the audit becomes a refactor. Fix: deliver the inventory and recommendations, then stop. Fixes are a separate, approved task.
• Fixing cheap redundancy first. Merging two harmless duplicate jobs while the revenue-critical webhook stays broken. Fix: order fix-next by business signal, not by ease.