Agentic AI Cost Control: When One Task Costs Forty Calls

By the CloudFinOpsKit team. 11 min read.

The moment you put a model in a loop, the unit of cost stops being a call. A chatbot turn is one request and one reply, and per-call accounting describes it perfectly well. An agent that plans, calls tools, reads results, and decides what to do next turns a single user request into dozens of model calls — and the relationship between those calls is what makes the arithmetic surprising.

Agentic spend is not simply "more calls". It is a different cost shape, with a growth curve most teams do not discover until an invoice arrives. This article is about that shape, the four things that drive it, the failure modes that turn it into a genuine incident, and the controls that bound it without crippling the agent.

The one thing to understand: context replay is quadratic

Models are stateless. An agent that "remembers" what it did three steps ago remembers because the orchestrator re-sends the entire history on every step.

Follow the money through a ten-step run:

• Step 1 sends the system prompt, the tool schemas, and the task.
• Step 2 sends all of that, plus step 1's reasoning and step 1's tool result.
• Step 10 sends all of it — every prior thought and every prior tool result — one more time.

Each step is billed for everything that came before it. Cumulative input tokens across a run therefore grow with roughly the square of the step count, not linearly with it. Doubling the number of steps an agent takes does not double its cost — it roughly quadruples it.

This is the single most important fact in agentic FinOps, and it explains a pattern that otherwise looks irrational: an agent that usually finishes in six steps is cheap, and the same agent on a slightly harder problem that takes it to eighteen steps is not three times more expensive — it is dramatically more expensive. The cost distribution across tasks has a long, heavy tail, and your average badly understates your risk.

It also tells you where to aim. Anything that shortens a run, or keeps the replayed history small, pays back super-linearly. That is a different instinct from classic token optimisation, where savings are proportional.

The other three multipliers

Tool schemas, billed on every turn

The full JSON Schema for every attached tool is part of the request on every step. An agent with fifteen tools pays for fifteen schema definitions per step, for the entire run, whether it uses them or not. On a twenty-step task that is three hundred schema transmissions to make fifteen tools available.

Attaching tools by phase — retrieval tools during research, write tools only once the plan is approved — cuts this directly, and reliably improves tool-selection accuracy at the same time. Fewer choices, fewer wrong choices.

Tool results, unbounded by default

This is the most common single cause of an agent cost blowout, and the easiest to fix.

An agent calls an API. The API returns 200KB of JSON. That result is inserted into the conversation verbatim — and then, because of context replay, re-sent on every subsequent step for the rest of the run. One careless tool call poisons the whole remaining task. A database query with no LIMIT, a file read with no size check, a search endpoint returning full documents instead of snippets: each one converts a cheap run into an expensive one.

Truncate at the tool boundary, not in the prompt. The orchestrator should cap what any tool may contribute to context, summarise or page anything larger, and make the truncation visible to the model so it can ask for more if it genuinely needs it.

Fan-out and sub-agents

Multi-agent designs multiply everything above. Each sub-agent carries its own system prompt, its own tool schemas and its own growing history, and the parent pays again to read the sub-agents' summarised results. A supervisor spawning five researchers is running six quadratic curves concurrently.

Fan-out is often the right design — it is genuinely faster and sometimes genuinely better — but it should be a deliberate choice with a budget attached, not the default topology for every task.

The failure modes that actually cost money

Steady-state agent cost is usually manageable. Incidents are what produce the invoice nobody expected, and they follow recognisable patterns:

• The stuck loop. The agent cannot make progress, retries a variation of the same step, and accumulates history — at quadratic rates — until something external stops it. With no step cap, "something external" is often the monthly bill.
• The retry storm. Throttling (HTTP 429) triggers client retries; the retries add load; the added load triggers more throttling. You are billed for the attempts. This is the agentic version of a thundering herd, and backoff with jitter is the fix.
• Reasoning on trivial steps. Extended reasoning is valuable on a hard planning step and pure waste on "parse this date". Reasoning tokens bill at output rates, so applying a high reasoning effort uniformly across every step of a run is expensive in exactly the places it adds nothing.
• The unbounded ingest. As above — one oversized tool result, replayed for the rest of the task.
• The abandoned run. A user closes the tab; the orchestrator does not notice; the agent finishes the task anyway and bills for an answer nobody will read.

Controls: defence in depth, because one limit is never enough

Each control below fails in different circumstances, which is precisely why you want several. A per-call max_tokens does nothing about a loop, because the loop decides how many calls there are.

Hard limits — the ones that stop the bleeding

1. A token budget for the whole task. The orchestrator accumulates input plus output tokens across every step, retry and sub-agent under one task identifier, and halts when the budget is spent. This is the control that actually bounds cost; everything else is refinement. Return a partial result with an explanation rather than failing silently.
2. A step cap. An independent ceiling on iterations. Catches the stuck loop even if the token accounting is wrong.
3. A wall-clock timeout. Catches the cases the other two miss — a hung tool, an abandoned run.
4. Tool-result caps. A maximum contribution to context per tool call, enforced at the boundary.
5. Provider-side quota. Rate limits and spend caps on the deployment itself, as a backstop for bugs in all of the above.

Efficiency levers — the ones that lower the curve

1. Compact the context. When history crosses a threshold, summarise the early portion and carry the summary forward instead of the transcript. This is the direct counter to the quadratic term, and it is what separates agents that can run long from agents that cannot.
2. Tier the model by step. Routing, summarising, formatting and extraction do not need the frontier model that the planning step needs. Per-step model selection is usually the largest saving available in a multi-step system, and the quality risk is contained because each step is narrow.
3. Cache the stable prefix. System prompt and tool schemas first, volatile history last, with the cache breakpoint between them. On a long run this is substantial — but note what it does not fix: the history itself changes every step, so caching lowers the rate on the fixed part without flattening the growth curve.
4. Scale reasoning effort to the step. High effort where the problem is genuinely hard, minimal elsewhere.
5. Make retries idempotent. A failure at step twelve should resume from step twelve, not restart the task. Re-running eleven successful steps to reach the failure is the most avoidable spend in the whole system.
6. Prune tools by phase. As above — smaller schemas, better tool choices.

Account for it properly: cost per task

Per-call metrics actively mislead for agentic systems. Average cost per call can fall while the cost of accomplishing anything rises, simply because the agent is making more, smaller calls. Two changes fix the accounting:

A task identifier that survives everything. Every model call, retry, sub-agent and tool invocation belonging to one user request carries the same ID. Without it you cannot tell a cheap task from an expensive one, and every aggregate is an average over things that are not comparable.

Distribution, not just mean. Because the tail is heavy, report the median and a high percentile. A median of a few cents with a 99th percentile a hundred times higher is a completely different business from a uniform cost per task — and only the percentile tells you what a bad week looks like. Track the tail as a first-class metric; it is where both the money and the reliability problems live.

From there, the metric that matters is cost per successful task: total spend, including every failed and abandoned run, divided by tasks that met the bar. An agent that fails a third of the time is not a third more expensive — it is a half more expensive, and the failed runs were often the longest.

Govern it before it ships

Agent budgets belong in the design review, not the incident review. Three things are worth making non-negotiable:

• No agent reaches production without a task budget, a step cap and a timeout. Treat it like a resource limit on a container — unremarkable, expected, and checked.
• A pre-production cost test. Run a representative sample of tasks, including deliberately hard ones, and record the distribution. The hard cases set the budget; the easy ones tell you nothing about your exposure.
• Alert on cost per task, not total spend. Total spend rising with adoption is success. Cost per task rising means something regressed — a prompt change, a chattier model, a tool returning more data than it used to — and that is the signal worth waking someone for.

Where to start

If you run agents today and have none of this, the order is: tool-result caps first (cheapest fix, removes the most common blowout), then a task budget and step cap (bounds the worst case), then context compaction and per-step model tiering (lowers the steady-state curve). Instrument cost per task alongside, because you cannot tell whether any of it worked without that number.

And know the baseline underneath it. Before optimising how an agent spends tokens, it is worth confirming you are not also paying for a deployment nobody calls, a provisioned-throughput allocation running at a fraction of its capacity, or a frontier model doing work a small one could do. The CloudFinOpsKit FinOps Agent reads 30 days of real token, request and failure metrics per deployment across Azure OpenAI, Amazon Bedrock and Vertex AI, prices them against actual billed cost, and flags exactly those patterns — including the token spikes and creep that a runaway agent produces. Fix the estate-level waste first; it is cheaper than any prompt change and it does not need a quality test.

FAQ

Why do agents cost so much more than a single call?

Because every step re-sends the accumulated history, so cumulative input tokens grow roughly with the square of the step count. Tool schemas billed each turn and unbounded tool results compound it.

How do I put a budget on an agent?

A hard token budget for the whole task, accumulated across every step and sub-agent under one task ID — plus an independent step cap and a wall-clock timeout. A per-call max_tokens does not bound a loop.

What is the right unit of cost?

Cost per completed task, reported as a distribution rather than a mean, and ultimately cost per successful task. Per-call averages can improve while the system gets more expensive.

Does prompt caching fix it?

It lowers the rate on the stable prefix, which is real money on long runs, but it cannot cache the history — that changes every step. You still need compaction, tool-result truncation and a task budget.

Related reading: tokenization: why the same meaning costs different amounts · cost per successful output · detecting AI cost anomalies · the token efficiency framework · FinOps for AI: the complete framework