Cost management works when it becomes a normal product discipline rather than a quarterly cost-cutting exercise. The end state I want is a shared operating system: instrumentation, budgets, routing, pricing, alerts, reviews, and a repeatable way to decide whether new AI capability is worth its marginal cost.
Weekly operating review
Architecture checklist
Cost scorecard
Launch gates
A staged implementation plan
The core cost equation
Start with cost per successful business outcome, not provider spend in isolation.
Include model calls, retries, tool/API fees, variable compute, storage/egress where meaningful, and required human review.
Keep fixed engineering and general platform overhead separate from the variable workflow model unless you deliberately need fully-loaded economics.
Instrumentation
Every cost event is attributable to organisation/customer, feature, workflow/run, provider, model, and environment.
Record usage units, estimated cost, success/failure, retry reason, and BYOK/platform-funded status.
Reconcile the internal ledger against provider invoices and investigate persistent variance.
Runtime controls
Route each task to the lowest-cost model that reliably satisfies its quality requirement.
Manage context growth with retrieval, compaction, bounded tool output, and durable state outside the prompt.
Give every autonomous run budgets for turns, spend, tools, retries, and elapsed time.
Cache or incrementally process work when inputs have not materially changed.
Customer controls
Budgets and entitlements exist per organisation or customer, not only at provider-account level.
Soft limits warn; hard limits prevent uncapped liability; authorised overrides are explicit and logged.
High-cost actions expose an understandable allowance impact or confirmation when that changes user behaviour.
Commercial model
Pricing reflects variable cost through included usage, tiers, paid expansion, BYOK, or another deliberate mechanism.
Review gross margin by plan, customer cohort, feature, and heavy-usage tail rather than only in aggregate.
Forecast from operational drivers such as active customers, runs per customer, workflow mix, model routing, and retry rates.
Operating cadence
Daily or near-real-time: watch spend velocity, runaway runs, abuse, and provider incidents.
Weekly: review cost per successful workflow, retry/failure spend, model-routing performance, top-cost tenants, and anomalies.
Monthly: reconcile invoices, review gross margin by cohort, update forecasts, and revisit plan allowances where behaviour has changed.
Before launches: estimate the new feature's variable cost, define limits, and decide what metric proves the extra spend creates value.
Staged implementation plan
Stage 1: Instrument every model call with customer, workflow, model, usage, and estimated cost.
Stage 2: Aggregate to cost per successful workflow and identify failures/retries separately.
Stage 3: Add per-run budgets, retry limits, context controls, and model routing.
Stage 4: Add organisation-level allowances, plan entitlements, abuse controls, and usage UI.
Stage 5: Connect usage to pricing, contribution margin, forecasting, and invoice reconciliation.
Stage 6: Run an ongoing cost review with clear owners and launch gates for expensive new capabilities.
Questions I would ask before approving an AI feature
What business outcome is this feature supposed to improve?
What does one successful use cost at low, typical, and worst-case usage?
What prevents a single request or customer from creating unbounded spend?
Can a cheaper model or deterministic system handle part of the workflow?
How does the product know the result was good enough to justify the cost?
What happens to gross margin if usage is ten times higher than forecast?
What alert tells us quickly if the economics change after launch?