# Your Agent Costs 10x More in Production Than in the Demo

> Where the money actually goes when an agent leaves the prototype: retries, context growth, judge models, and the human review nobody budgeted.

Published: 2026-03-10  
Topic: Agent engineering  
Author: Alex Cinovoj, Founder, TechTide AI  
Canonical: https://clawli.ai/blog/agent-cost-in-production

The prototype cost eleven cents a run. The invoice said otherwise. Here is the arithmetic nobody does before shipping.

## Key takeaways

- Production multiplies cost through retries, longer context, and failure paths the demo never hit.
- Evaluation is a real line item. Judge models and human review often exceed inference.
- Per-run cost accounting is the only fix that survives; sampling averages hide the tail.
- Cheap models on the boring 80% is the highest-return optimization available.

Every agent has two costs: the one in the pitch deck and the one on the invoice. The gap is not vendor pricing. It is that a demo runs the happy path with a short history against a clean input, and production does none of those things.

## Where the multiplier comes from

- **Retries.** A 6% tool-failure rate with two retries is a 12% surcharge on every run that touches that tool, and the retry carries the full accumulated context.
- **Context growth.** Session five looks nothing like session one. A conversation that starts at 4k tokens and ends at 60k costs 15x on the last call alone.
- **Failure paths.** Runs that abandon still bill. Nobody counts them because they produced nothing, which is precisely why they hide.
- **Judges.** Automated evaluation runs a second model over the first model’s output. That is not overhead you can skip, and it is not free either.
- **Humans.** Review time is the largest line item at most companies and the one that never appears in a token dashboard.

Independent write-ups of 2026 pricing land on the same order of magnitude. Production burn commonly 5-15x the prototype estimate ([Edgeless Lab](https://edgelesslab.com/blog/real-cost-ai-agents-production-2026/)), and evaluation costs that are rarely a single line item ([ContextQA](https://contextqa.com/blog/real-cost-of-ai-agent-evaluation/)). Our own numbers sit in that band.

> **The number that matters**
>
> Not cost per token. Not cost per run. **Cost per accepted output.** A run that gets rejected in review cost you twice: once in inference, once in the reviewer’s attention.

## Instrument first, optimize second

We write a cost event at the end of every agent run: model, prompt tokens, completion tokens, tool calls, wall time, outcome. It took an afternoon and it changed what we worked on for the next quarter, because the distribution was nothing like the average.

The p50 run was cheap and boring. The p99 run cost 40x the median and was almost always the same shape: a tool returning junk, the model retrying, context ballooning, no stopping condition. Fixing one tool’s error handling removed more spend than any model swap would have.

## The optimizations that actually paid

1. **Route by difficulty.** Classification, extraction, formatting, and routing go to a small model. Only the load-bearing generation gets the expensive one. This is usually a 60-70% reduction and costs almost no quality.
2. **Cap the loop.** A hard step limit and a hard token budget per run. When it trips, fail loudly and surface it, a visible failure is cheaper than an invisible ramble.
3. **Summarize tool output.** Three fields instead of a 40KB payload. Cuts input tokens on every subsequent call in the run.
4. **Cache the stable prefix.** Identity and profile blocks do not change between runs; prompt caching turns them into a rounding error.
5. **Kill the retries you cannot justify.** If a tool fails twice, the third attempt is superstition.

## What we refuse to optimize

Research depth. It is tempting, because research is the most token-hungry stage and the one whose output the customer never sees directly. It is also the stage that produces the proprietary detail that makes a post worth publishing at all. Cutting it saves money and destroys the product.

> Cheap output that nobody reads is not a cost saving. It is a slower way to spend the same money.

## Budgets as a product feature

Once per-run cost is real, it becomes something you can expose. Per-organization daily token limits tied to plan, a visible spend meter, and a clear message when a limit trips beats a surprise invoice for everyone involved. It also converts an unbounded liability into a bounded one, which is the difference between a hobby and a business.

Related: [token budgets are the new headcount](/blog/token-budgets-are-the-new-headcount) and [multi-agent systems fail quietly](/blog/multi-agent-observability).

The full cost model, including the spreadsheet, is on [my site](https://alexcinovoj.com/). If you want someone to run the numbers against your own workload, that is the sort of thing we do at [TechTide AI](https://techtideai.io/).

## FAQ

### What is a realistic cost per run for a writing agent?

It depends entirely on research depth. A single-pass draft with a small context is cents. A researched, tool-using, evaluated piece is dollars. The useful discipline is not a target number but per-run accounting so you can see your own distribution.

### Does self-hosting a model save money?

Only above a break-even volume that most teams never reach, and it moves cost from an invoice to an engineering team. Route to smaller hosted models first; that captures most of the saving with none of the operational load.

### How do I budget for evaluation?

Treat it as five separate lines: judge inference, tooling, human review, harness engineering, and maintenance as models drift. Teams that budget only for the first one are usually off by an order of magnitude.

### Should users see token costs?

Users should see limits and remaining capacity, not raw token counts. Tokens are an implementation detail; "you have 3 of 20 briefs left today" is information they can act on.

## Sources

- [Edgeless Lab. The Real Cost of Running AI Agents in Production (2026)](https://edgelesslab.com/blog/real-cost-ai-agents-production-2026/)
- [ContextQA. The Real Cost of AI Agent Evaluation](https://contextqa.com/blog/real-cost-of-ai-agent-evaluation/)
- [Anthropic. Prompt caching documentation](https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching)

Tags: cost, observability, operations