ai-provider-integration

AI Provider Integration

Treat every AI vendor as a swappable adapter behind one typed contract. The money and the data survive vendor failures only if the lifecycle, not the vendor SDK, is the source of truth.

1. Adapter interface

Every provider implements the same contract. No provider SDK types leak past the adapter.

interface ProviderAdapter {
  capabilities(): ProviderCapability[]
  estimate(request: GenerationRequest): ProviderCostEstimate
  submit(request: GenerationRequest, idempotencyKey: string): ProviderJobResult
  normalizeWebhook(rawPayload: unknown): ProviderJobResult
  verifyWebhook(headers: Headers, body: string): boolean
}

2. Job lifecycle

reserved -> submitted -> processing -> succeeded | failed | cancelled | expired -> reconciled

Order is strict:

1. Budget reservation exists before the job record.
2. Job record exists before the paid API call.
3. Every submit carries an idempotency key tied to the job record.
4. Webhook handlers check job state before writing. Replays must not double-debit.

3. Response validation

Parse every provider response through a schema before it touches your data model.

const parsed = providerResponseSchema.safeParse(raw)
if (!parsed.success) throw new ProviderSchemaError(provider.name, parsed.error)

Empty strings and nulls from a provider are data loss, not defaults. Fail the job, do not store them silently.

4. Cost tracking (reservation model)

estimate -> reserve -> submit -> [success: debit actual] | [failure: release]

Run a scheduled reconciliation job that finds reservations older than the provider's max job age and releases or flags them. Orphaned reservations are guaranteed, plan for them.

5. Retry and fallback classes

Classify first, then act. Never retry a 400.

6. Health monitoring

Circuit breaker per provider. Track success rate, p50/p95/p99 latency, error rate, cost per generation. Open the breaker on sustained failure, route to fallback, probe before closing.

Good vs bad

Bad: call the provider API, then create the job record from the response. A crash between the two burns money with no record.

Good: reserve budget, insert the job row in state reserved, then submit with an idempotency key. A crash at any point leaves a row the reconciler can settle.

Verification

Do this before calling the integration done: replay the same completion webhook twice against a finished job. Expect exactly one debit in the ledger and a no-op 200 on the second delivery. If the ledger shows two debits, the handler is missing the job-state check before the write. Add it and re-run.

Also kill the process between reserve and submit. Expect the reconciliation job to release the orphaned reservation within one cycle. If it does not, the reconciler's age filter or state filter is wrong.

Completion checklist

• [ ] All provider calls go through the adapter interface, no SDK types outside it
• [ ] Job row and budget reservation exist before any paid call
• [ ] Webhook signature verified and replay is idempotent
• [ ] Every response schema-validated before storage
• [ ] Retry logic branches on error class, 400s never retried
• [ ] Reconciliation job scheduled and tested against an orphaned reservation
• [ ] Circuit breaker with fallback provider wired for capacity failures

Any box unchecked: not done. Fix or say so.

Footguns

• Webhook replay double-debits. Providers redeliver. Fix: debit only on the state transition to succeeded, and make the transition conditional on current state.
• Trusting `estimate` as the final cost. Estimates drift from actuals. Fix: debit the actual reported cost on success, release the difference.
• Retrying permanent errors. A 400 retried 5 times is 5x the latency and sometimes 5x the cost. Fix: classify before retry, permanent fails fast.
• Storing empty provider output as success. Fix: schema rejects empty required fields, job goes to failed with the raw payload attached for debugging.