Skip to content
Orlix
All notes

[ observability ] · · 5 min

What your AI bill is hiding (and per-step cost reveals)

Provider invoices tell you what you spent. They can't tell you which step, which workflow, or which retry spent it. Traces can.

Every month, a finance team somewhere opens a model-provider invoice, sees a number 40% higher than last month, and asks engineering why. And engineering, if all they have is the invoice, genuinely cannot answer. The invoice aggregates by API key. The question is about workflows.

Cost is a property of a span

The unit that actually explains spend is the step within a run: this model call, with this input size, in this workflow version, on this retry. Attach cost there and the invoice becomes decomposable:

  • By workflow — contract-review costs 3× claims-intake per run; is it worth 3×?
  • By step — 71% of one customer's spend was a single summarisation step that later proved removable.
  • By retry — a flaky connector was silently tripling model calls; the model wasn't the problem, the retry policy was.
  • By version — v14 costs 22% more per run than v13. That's a fact you want before fleet-wide rollout, which is what budgets on canary deploys are for.

None of these questions is answerable from an invoice. All of them fall out of a trace.

Alert on trend, not breach

The second failure mode of cost management is timing. An alert that fires when the budget is breached fires after the money is gone. Projecting month-end spend from the trailing trend and alerting on the projection turns a post-mortem into a decision: the workflow that will exceed budget in eleven days is a workflow you can still change.

A useful budget policy has three lines:

  1. A per-run ceiling that fails the run (protects against pathological inputs).
  2. A per-workflow monthly projection alert (protects the invoice).
  3. A ranking of steps by contribution when either fires (protects the on-call's evening).

The uncomfortable part

Teams sometimes discover that a beloved workflow doesn't pay for itself. That's not a failure of observability — it's the point. The alternative isn't a cheaper workflow; it's the same bill with no idea which work it bought.

Spend is a design input. Give it the same per-step visibility you give latency, and it starts behaving like one.