The API Bill Is Not the Cost Model

The situation

AI costs usually look simple during an MVP. There may be one provider, one model, a small number of users, and one monthly invoice. As soon as the product becomes agentic or multi-step, that number stops being useful on its own. One user request can cause several model turns, tool calls, code execution, embeddings, image generation, retries, storage writes, browser sessions, and background jobs. A request that looks like one feature in the UI may actually be a chain of billable events underneath it.

The naive solution

Watch the provider dashboard and set a monthly company budget. That is better than nothing, but it is still accounting after the fact. It tells me that spend increased without telling me what behaviour created it or what I should change in the product.

The hidden constraint

The unit I pay for is rarely the unit the customer thinks they are buying. Providers may charge for tokens, cached tokens, images, audio seconds, storage, tool use, or compute. The customer thinks they bought an audit, a generated application, a support answer, a report, or a completed task. If I cannot connect those two layers, I cannot make a good pricing or product decision.

The decision

Treat AI cost as a workflow-level unit economics problem. I want to know the total variable cost required to produce one completed outcome, and I want that number broken down by customer, feature, model, and failure path.

The implementation

I would record a cost event for every model call and meaningful variable-cost action, then attach those events to a run or workflow ID. A useful baseline formula is: completed-workflow cost = model input + model output + tool/API fees + variable compute + storage/egress attributable to the run + retries/repair + any required human review. I would keep fixed engineering salaries and normal platform overhead separate at first so the variable economics remain visible.

The tradeoff

This adds instrumentation before it feels urgent. The temptation is to wait until spend is large. The problem is that once the bill is large, historical attribution is usually missing and the team has to guess which workflows are responsible.

The evidence I would look for

The signal I would look for is not simply a lower invoice. I want to be able to answer questions such as: what did one successful generation cost, what did failed generations cost, which customers consumed the most platform-funded AI, and which feature has the highest cost per successful outcome.

The reusable lesson

Measure AI in the unit the business sells, not only in the unit the provider bills. Tokens are an implementation detail; cost per useful outcome is the management metric.

The consultant's questions

• What does one completed customer outcome cost today, end to end?
• Which parts of that cost are model spend and which are tools, compute, storage, or human review?
• Can failed and cancelled work be separated from successful work?
• Can every pound or dollar of variable AI spend be attributed to a customer or internal workflow?