Concept
Cost telemetry
Tokens, dollars, latency, and quality, recorded per run and attributable to a specific post.
Agent systems fail quietly and expensively. The only defense is knowing what each unit of output cost before the invoice arrives.
What gets recorded
- Prompt and completion tokens per model call, with the model name.
- Estimated cost per call, derived from the model catalog's per-token pricing.
- Latency per span, so a slow run points at a slow tool rather than a slow model.
- Evaluation scores from the grading pass, joined to the same run.
Cost per post
Run-level numbers are aggregated to the artifact. The useful question is not what a model costs per million tokens, it is what one publishable post costs end to end, including the drafts that failed pre-flight and the research that produced nothing.
Budgets
Limits are enforced per organization before the model call, not after. A run that would exceed the daily budget is refused with a clear message rather than truncated halfway through.