Can you replay a failure?
Pick a failed run. Check whether you can reconstruct its inputs, tool calls, and model version. Record what is missing before trying to reproduce the failure.
Decision framework
Should this AI workflow ship? Start with seven questions.
Use these questions to review an AI workflow with the people who build and operate it. Bring logs, test results, and usage costs.
My background spans 13 years in US enterprise IT. These checks reflect the operational questions I bring to AI work: who owns it, what can go wrong, and how the team will respond.
The seven checks
Answer each one out loud, with evidence.
Do evals come from real failures?
Include known failures alongside representative and unusual cases. Check that a prompt or model change does not bring an old failure back.
What is the blast radius of a wrong call?
List every irreversible action the system can take: sends, refunds, writes, deletes. Each one needs either a confirmation gate or a compensating action. Unbounded write access is not a launch state.
Can you attribute cost to a workflow?
Track usage by workflow and, where relevant, by customer. Set limits and alerts so repeated tool calls cannot consume an unchecked budget.
Is untrusted text separated from instructions?
Treat retrieved documents, tickets, and web pages as untrusted input. Test whether instructions inside them can change the agent’s behavior or access.
Who owns it on Monday?
Assign an owner for alerts, access changes, and maintenance. Give that person the documentation and permissions needed to operate the system.
What is the fallback when the model is down?
Providers degrade. A queue, a cached answer, a human path, or a clear failure message. Something must exist between the model and your customer.
Reading the result
Four possible recommendations
Ship
The evidence supports a limited release, with the important risks addressed. Agree what to monitor and when to pause.
Fix
The workflow is useful, but specific gaps need attention. List the changes, their owners, and the checks that will confirm they work.
Redesign
Change the approach when the current design cannot meet the requirements. Compare a smaller workflow, different tools, or a manual step before rebuilding.
Stop
Pause or end the work when the expected benefit does not justify its cost or risk. Document the reason and what would need to change.
FAQ
Common questions.
- Is this a paid offer?
- No. This page is the framework itself, written out, and you are welcome to run it yourself. The paid version is the $1,000 Systems Decision Audit at TechTide AI.
- How long does a review like this take?
- Running it yourself depends on the workflow and the evidence available. The paid Systems Decision Audit is a 90-minute live review, with a written verdict within 48 hours.
- Who should run it?
- Anyone with access to the traces, the eval results, and the bill. If none of those exist, that is already the finding.
- What if every check fails?
- Pause deployment and assess the risk of each gap. A failed permission check can matter more than several lower-risk checks combined.
- What if the problem is not the AI?
- Frequently it is not. When the real constraint is the data or the process underneath, fix that first. The AI work is downstream of it.
Ran the checks. Now what?
If the failures sit in the AI system itself, the paid version of this framework is the Systems Decision Audit, $1,000. A 90-minute live review of one stalled AI workflow: inputs, retrieval, tool permissions, failure handling, security posture. Written verdict within 48 hours — Ship, Fix, Redesign, or Stop. Full fee credited toward implementation if it starts within 30 days.
Working code and technical write-ups live on alexcinovoj.dev. More of the reasoning behind these checks is in the field notes.