Agent engineering
Your Agent Costs 10x More in Production Than in the Demo
The prototype cost eleven cents a run. The invoice said otherwise. Here is the arithmetic nobody does before shipping.
Key takeaways
- Production multiplies cost through retries, longer context, and failure paths the demo never hit.
- Evaluation is a real line item. Judge models and human review often exceed inference.
- Per-run cost accounting is the only fix that survives; sampling averages hide the tail.
- Cheap models on the boring 80% is the highest-return optimization available.
Every agent has two costs: the one in the pitch deck and the one on the invoice. The gap is not vendor pricing. It is that a demo runs the happy path with a short history against a clean input, and production does none of those things.
Where the multiplier comes from
- Retries. A 6% tool-failure rate with two retries is a 12% surcharge on every run that touches that tool, and the retry carries the full accumulated context.
- Context growth. Session five looks nothing like session one. A conversation that starts at 4k tokens and ends at 60k costs 15x on the last call alone.
- Failure paths. Runs that abandon still bill. Nobody counts them because they produced nothing, which is precisely why they hide.
- Judges. Automated evaluation runs a second model over the first model’s output. That is not overhead you can skip, and it is not free either.
- Humans. Review time is the largest line item at most companies and the one that never appears in a token dashboard.
Independent write-ups of 2026 pricing land on the same order of magnitude. Production burn commonly 5-15x the prototype estimate (Edgeless Lab (opens in a new tab)), and evaluation costs that are rarely a single line item (ContextQA (opens in a new tab)). Our own numbers sit in that band.
Instrument first, optimize second
We write a cost event at the end of every agent run: model, prompt tokens, completion tokens, tool calls, wall time, outcome. It took an afternoon and it changed what we worked on for the next quarter, because the distribution was nothing like the average.
The p50 run was cheap and boring. The p99 run cost 40x the median and was almost always the same shape: a tool returning junk, the model retrying, context ballooning, no stopping condition. Fixing one tool’s error handling removed more spend than any model swap would have.
The optimizations that actually paid
- Route by difficulty. Classification, extraction, formatting, and routing go to a small model. Only the load-bearing generation gets the expensive one. This is usually a 60-70% reduction and costs almost no quality.
- Cap the loop. A hard step limit and a hard token budget per run. When it trips, fail loudly and surface it, a visible failure is cheaper than an invisible ramble.
- Summarize tool output. Three fields instead of a 40KB payload. Cuts input tokens on every subsequent call in the run.
- Cache the stable prefix. Identity and profile blocks do not change between runs; prompt caching turns them into a rounding error.
- Kill the retries you cannot justify. If a tool fails twice, the third attempt is superstition.
What we refuse to optimize
Research depth. It is tempting, because research is the most token-hungry stage and the one whose output the customer never sees directly. It is also the stage that produces the proprietary detail that makes a post worth publishing at all. Cutting it saves money and destroys the product.
Cheap output that nobody reads is not a cost saving. It is a slower way to spend the same money.
Budgets as a product feature
Once per-run cost is real, it becomes something you can expose. Per-organization daily token limits tied to plan, a visible spend meter, and a clear message when a limit trips beats a surprise invoice for everyone involved. It also converts an unbounded liability into a bounded one, which is the difference between a hobby and a business.
Related: token budgets are the new headcount and multi-agent systems fail quietly.
The full cost model, including the spreadsheet, is on my site (opens in a new tab). If you want someone to run the numbers against your own workload, that is the sort of thing we do at TechTide AI (opens in a new tab).
“You are not paying for tokens. You are paying for every time the system did not know when to stop.”
Questions people actually ask
- What is a realistic cost per run for a writing agent?
- It depends entirely on research depth. A single-pass draft with a small context is cents. A researched, tool-using, evaluated piece is dollars. The useful discipline is not a target number but per-run accounting so you can see your own distribution.
- Does self-hosting a model save money?
- Only above a break-even volume that most teams never reach, and it moves cost from an invoice to an engineering team. Route to smaller hosted models first; that captures most of the saving with none of the operational load.
- How do I budget for evaluation?
- Treat it as five separate lines: judge inference, tooling, human review, harness engineering, and maintenance as models drift. Teams that budget only for the first one are usually off by an order of magnitude.
- Should users see token costs?
- Users should see limits and remaining capacity, not raw token counts. Tokens are an implementation detail; "you have 3 of 20 briefs left today" is information they can act on.
Sources & further reading
Written by
Alex Cinovoj, Founder, TechTide AI
Alex Cinovoj builds ClawLI at TechTide AI, a fleet of specialist agents that research, draft, schedule, and grade LinkedIn content with a human approving every send. He writes about what breaks when you put agents in front of a real audience.
Keep reading
- Agent engineeringMulti-Agent Systems Fail QuietlyMulti-agent systems degrade without erroring. Traces, spans, and the telemetry that turns an inexplicable bad output into a fixable bug.
- Agent engineeringPre-Flight: The Checks That Run Before an Agent PublishesNine deterministic checks that catch the errors a model will never catch about itself, running in the last second before publish.
- Agent engineeringEvals or It Didn’t ShipLLM evals for agents that write, when there is no correct answer. Golden sets, rubric judges, and the metrics that survive contact with reality.