Why the Average Cost of an AI Agent May Tell You Almost Nothing
Suppose two AI agents perform the same business task. Agent A costs $4.80 on average. Agent B costs $4.50. Easy decision, right? Choose Agent B.
Now add one more piece of information: most Agent A executions cost between $4 and $6, with expensive cases reaching $8. Most Agent B executions cost only $2 or $3—but occasionally it enters a long reasoning or retry loop and costs $30, $40, or $50.
Still choosing B?
This reveals a key consequence of agentic economics:
Two AI agents with similar average costs can have completely different economic risk.
And that means average cost may be telling management far less than it thinks.
An AI Agent Doesn't Have One Cost
Agentic systems don't necessarily follow the same computational path each time they receive the same task. They reason differently, invoke different tools, accumulate varying context, retry failures, or spend additional inference verifying their work.
McKinsey cites research showing as much as a 30× difference in cost between completions of the same programming task and emphasizes that agentic cost behaves as a distribution rather than a fixed unit price.
A 2026 study examining 2,700 agent executions found that while task specifications affect token consumption, significant run-to-run variance remains—requiring models that predict the distribution of token expenditures rather than a single expected number.
Traditional Question: “What should this task cost?”
Agentic Question: “What range of costs should we expect this task to produce?”
The Average Can Be Correct and Still Mislead You
Imagine a dashboard displaying a green status indicator for Average cost per completed task: $3.18. Everything looks fine until you examine the underlying distribution:
- Median (P50): $1.90
- P90: $4.75
- P95: $9.20
- P99: $31.40
- Maximum: $187.00
The average wasn't wrong—it simply concealed where the operational risk lived. At enterprise scale, a small tail compounds quickly: a workflow running 10 million times with a 1% expensive-tail rate produces 100,000 tail events.
At enterprise scale, the tail eventually becomes somebody's budget.
The Tail May Be Telling You Something
Expensive executions are rarely random. A growing P99 cost often serves as a diagnostic signal indicating underlying systemic issues:
- Repeated tool failures or retry storms
- Runaway reasoning loops and unconstrained context expansion
- Unnecessary model escalation or duplicate tool invocations
- Inefficient memory retrieval and weak termination rules
AWS treats these as primary agentic cost risks, recommending tracking cost at the reasoning-cycle, agent, workflow, and completion levels rather than relying on aggregate cloud bills. When P99 spikes, the critical question is not just “How do we cut spending?”, but “What changed in the behavior of the system?”
Average Cost and Operating Risk Are Different Things
Just as in financial markets or manufacturing yield management, two systems can share an expected mean while possessing vastly different downside risks:
Agent A: Average: $4.80 | P99: $8.00 (Wins on Predictability)
Agent B: Average: $4.50 | P99: $45.00 (Wins on Mean Cost)
Whether Agent B is acceptable depends on business outcomes: do those expensive executions correspond to unusually complex, high-value tasks, or are they caused by broken tool integration? The distribution reveals where to look; the business outcome determines whether to care.
From Cost Monitoring to Cost Diagnosis
A mature enterprise AI cost dashboard must evaluate the full distribution lifecycle:
Expected Cost (Mean) → Typical Cost (Median) → Tail Percentiles (P95/P99) → Tail Frequency → Behavior Diagnostics → Business Value Verification
AWS recommends percentiles such as P50, P90, and P99 alongside per-task budgets and automatic cutoffs. Connecting distribution metrics to business diagnostics converts cost monitoring into actionable operational governance.
The Mean Is Only the Beginning
Expected cost remains an important metric, but once autonomous systems take variable paths to complete work, the average stops being the whole economic story. Variance, tail behavior, and execution patterns dictate true operational cost.
The cheapest agent on average may not be the cheapest agent to operate. The core executive question shifts from “What is our average cost per AI task?” to:
What is our P99 AI outcome costing us—and do we know why?