When AI Costs Stop Behaving Like Software Costs

Software has always had variable costs: more users consume more compute, more traffic requires more infrastructure, and more API calls create larger bills. But agentic AI introduces something stranger.

The same task, performed by the same kind of system, can take a radically different computational path—and incur a radically different cost.

Researchers studying eight frontier models on agentic coding tasks found token consumption to be highly variable and stochastic. Repeated runs of the same task could differ by as much as 30× in total token usage. Even more surprisingly, spending more did not reliably produce better results—accuracy often improved to a point and then flattened.

The cost of agentic software increasingly depends not only on what task you ask it to perform, but on what happens while it tries to perform it.

---

The Journey Now Has a Price

Traditional software gives us a fairly intuitive relationship:

More Work → More Resources → More Cost

Agentic AI adds another dimension:

Execution Path A: Reason → Call Tool → Answer

Execution Path B: Reason → Search → Inspect → Call Tool → Fail → Retry → Retrieve Context → Revise → Verify → Answer

Same business request. Very different journey.

McKinsey points directly to this problem: agents can take different paths, invoke different tools, and retry in different ways. In one programming example, completion costs for the same task varied by a factor of 30.

Their conclusion is unusually important: cost behaves as a distribution, and average-cost budgets may not be enough. This isn't merely usage-based software—it is behaviorally variable software.

Apparently, the Agent Doesn't Know Either

Budgeting usually starts with the task: we know approximately what needs to happen, estimate the resources required, and assign a reasonable expected cost. Agentic work makes each step less certain—we may not know which reasoning path the agent will take, how many tools it will call, how much context it will accumulate, or whether one failed attempt becomes three.

Perhaps most surprisingly, the models themselves aren't particularly good at predicting this. In the same agentic-coding study, frontier models systematically underestimated their eventual token consumption, with the correlation between predicted and realized usage reaching only about 0.39 at best.

The software knows what task it has been given. It may not know what that task is going to cost.

The Expensive Part May Be Deciding You're Done

Here is where the story gets more interesting. A separate study of multi-agent software development found that iterative code review consumed an average of 59.4% of total tokens.

The expensive part wasn't necessarily creating the work—it was deciding whether the work was good enough:

Create → Inspect → Revise → Re-inspect → Retry → Verify

Each step makes sense: if the answer looks uncertain, check it; if the tool fails, retry; if still not confident, add context; if needing more assurance, verify again. But every additional attempt costs money.

AWS is already warning about this in production systems. Its Agentic AI guidance identifies excessive tool calls and "failed-invocation retry storms" as sources of unpredictable cost spikes. A retry is no longer merely an engineering decision—it is an economic one.

Average Cost Can Hide the Problem

Imagine two agents doing the same work:

Agent A: Costs $4.80 on average. Most runs fall between $4 and $6; expensive cases reach $8.

Agent B: Averages only $4.50. Most runs cost $2 or $3, but occasionally a reasoning-and-retry loop pushes the cost to $30, $40, or $50.

Which agent is cheaper? The average says B, but an operations leader might disagree. Once cost behaves as a distribution, management needs more than the mean. It needs to know:

• How widely costs vary
• What the expensive tail looks like
• What causes those expensive runs
• How frequently they occur
• Whether the additional spending improved the outcome

This begins to look less like traditional software budgeting and more like manufacturing yield, reliability engineering, or financial risk.

“Retry until correct” sounds prudent, but economically it isn't a strategy. The better rule is closer to:

Keep spending while the expected improvement is worth the expected additional cost.

That is economics entering orchestration.

The Budget Becomes Part of the Architecture

This may be the real shift. Traditionally, the budget sits above the software system—management approves spending, watches the bill, and intervenes when costs drift too far.

Agentic systems may require some of that economic logic inside the workflow itself:

• How many retries are allowed?
• When should the agent switch to a cheaper model?
• When is another verification pass worth paying for?
• When should an expensive exception escalate to a human?
• When should the system simply stop?

Those sound like technical configuration choices, but they are also decisions about spending authority.

McKinsey argues that organizations should measure agentic work against completed business outcomes rather than tokens or model calls alone. That is right—but it may need one more dimension: not just “What is the average cost per completed outcome?”, but:

What is the distribution of possible costs required to produce that outcome?

Because once software can reason longer, retrieve more information, call another tool, or try again while performing the work, cost control cannot happen only after the invoice arrives. The budget becomes part of the architecture.

And that leaves executives with a question traditional software rarely forced them to ask:

Who decides how much the software is allowed to spend trying to succeed?