How should I forecast AI costs?

Forecast accepted outputs per month and multiply by cost per accepted output, including retries and eval calls. Volume forecasts on tokens are unstable because context size drifts.

What is a reasonable cost per accepted output?

It depends entirely on the human alternative. The useful comparison is the loaded hourly cost of the person who would otherwise do it, not another vendor’s token price.

How do I stop a runaway agent loop?

Per-run token and step ceilings enforced in the loop itself, not just monitoring. Monitoring tells you afterwards; a ceiling prevents the invoice.

Is a cheaper model always worth trying?

On non-reasoning steps, almost always. Validate with a golden set rather than intuition. The quality difference on extraction and formatting is usually nil.

Journal

Go-to-market

Token Budgets Are the New Headcount

Finance understands headcount. It does not yet understand why your inference bill tripled in a month with no new customers. Here is the translation layer.

Alex Cinovoj TechTide AIJune 2, 20263 min read

Key takeaways

Allocate budget per workflow, not per team. Workflows are what actually consume tokens.
Cost per accepted output beats cost per token, run, or user for every planning conversation.
Retries and long contexts are the two silent multipliers; both are controllable.
A hard cap with graceful degradation prevents the incident that ends the programme.

The first time an AI feature’s cost curve surprises a finance team, the reaction is not "let us optimize." It is "turn it off until we understand it." Several genuinely good products have died in that meeting, not because the economics were bad but because nobody could explain them in the language the room used.

Translate to something already understood

Headcount is legible because it maps to output: this person produces roughly this much work. Tokens are not legible to anyone outside the team. So do the conversion before the meeting.

cost per accepted output =
 (total inference spend for the workflow)
 ÷ (outputs a human approved and shipped)

Include: retries, failed runs, judge/eval calls, embeddings.
Exclude: nothing. Rejected work is part of the cost of accepted work.

Once you have that number you can say something like: this workflow produces an approved post for £2.40 including everything it wasted getting there, against roughly forty minutes of a specialist’s time. That is a sentence a board understands.

Budget per workflow

Team-level budgets fail because a single expensive workflow buried in one team’s allocation looks fine until it does not. Workflow-level allocation makes cost a property of the thing that causes it.

Every workflow gets a monthly ceiling and a per-run ceiling.
Per-run ceilings catch the runaway loop within minutes rather than at month end.
Monthly ceilings degrade gracefully: fall back to a smaller model, then queue, then stop with a clear message.
Every run writes a cost event with model, tokens, and workflow. No exceptions, or the whole scheme is fiction.

The two silent multipliers

Retries. A three-attempt retry policy on a step that fails 20% of the time is a manageable 1.2x. On a step that fails 60% because of a bad tool contract, it is a 2.4x that nobody notices because each individual run looks normal. Track retry rate per step as a first-class metric, not a log line.

Context growth. Every added instruction, example, and retrieved memory is paid for on every call forever. Context has a way of accreting (a paragraph here, another few-shot example there) and a prompt that started at 2k tokens is at 14k by summer with no one having made a decision to double the bill. Budget context in tokens and review it monthly. See context engineering.

Nobody ever decided to triple the inference bill. Everybody added one more example to the prompt.

Model selection is a budgeting decision

Routing every step to the strongest available model is the single most expensive habit in production AI, and usually indefensible. Classification, extraction, routing, and formatting run fine on small models. Reserve the expensive one for the step where reasoning quality genuinely changes the output, typically one step in four.

The measurement is straightforward: swap the model on one step, run your golden set, and look at whether the binary criteria hold. If they do, the cheaper model wins and the argument is over.

Defending the budget

Bring three numbers: cost per accepted output with trend, the human-time equivalent, and the ceiling that guarantees no surprise. That third one is what actually gets the budget approved. Finance is not opposed to spend; it is opposed to unbounded spend it cannot forecast.

The budgeting template, including the per-team allocation math, is on Alex Cinovoj (opens in a new tab). We run the same model across client programs at TechTide AI (opens in a new tab), and it has never once come in where the demo suggested.

“Cost per accepted output is the only AI metric a CFO has ever been glad to see.”

Alex Cinovoj, TechTide AI

Questions people actually ask

How should I forecast AI costs?: Forecast accepted outputs per month and multiply by cost per accepted output, including retries and eval calls. Volume forecasts on tokens are unstable because context size drifts.
What is a reasonable cost per accepted output?: It depends entirely on the human alternative. The useful comparison is the loaded hourly cost of the person who would otherwise do it, not another vendor’s token price.
How do I stop a runaway agent loop?: Per-run token and step ceilings enforced in the loop itself, not just monitoring. Monitoring tells you afterwards; a ceiling prevents the invoice.
Is a cheaper model always worth trying?: On non-reasoning steps, almost always. Validate with a golden set rather than intuition. The quality difference on extraction and formatting is usually nil.

Sources & further reading

Written by

Alex Cinovoj, Founder, TechTide AI

Alex Cinovoj builds ClawLI at TechTide AI, a fleet of specialist agents that research, draft, schedule, and grade LinkedIn content with a human approving every send. He writes about what breaks when you put agents in front of a real audience.

alexcinovoj.com techtideai.io See the pre-flight gate