FinOps for AI: The Complete Framework
In two years AI went from a line item somebody was curious about to something nearly every practitioner is now responsible for. The FinOps Foundation's State of FinOps data tracks the shift bluntly: the share of practitioners managing AI spend went from roughly a third to effectively all of them. The Foundation made the change structural — FinOps for AI is now a technology category within the Framework, and the Foundation's mission was restated from advancing the people who manage the value of cloud to the value of technology.
That is the context. This page is the practical map: what genuinely differs about AI spend, how it maps onto the FinOps Framework you already run, the metrics worth tracking, a maturity ladder you can place yourself on, and a curated library of the authoritative guidance — so you are not assembling this from conference talks and vendor blog posts.
The four things that actually make AI different
Plenty of "AI is different" commentary amounts to "it is expensive and growing". That is not a structural difference; that is a big number. Four things are genuinely structural, and each one breaks a specific piece of standard FinOps machinery.
1. The billing unit is a virtual currency
Classic cloud cost is resource-time: an instance-hour, a provisioned IOPS, a gigabyte-month. AI cost is denominated in tokens — and tokens are a synthetic unit whose relationship to the work you asked for is not fixed. The same request can cost wildly different amounts depending on how the text was serialised, what language it is in, and how much context the framework attached. That is a property you control, which makes it an optimisation surface that has no equivalent in classic FinOps. It is also why tokenization deserves its own engineering attention.
This is exactly what FOCUS 1.2 addressed at the data layer, adding virtual-currency support so credits and tokens can be reported in the currency the vendor prices in and converted to a national currency. In practice that means four columns: PricingCurrency, PricingCurrencyEffectiveCost, PricingCurrencyListUnitPrice and PricingCurrencyContractedUnitPrice. Their significance is not administrative — it is what makes token consumption from different vendors comparable inside one dataset instead of a pile of incompatible exports.
2. Tokens cannot be tagged
This is the one that quietly defeats most allocation programmes. Your tagging policy, your management-group hierarchy, your cost-allocation rules — none of them reach inside a model deployment. A single Azure OpenAI deployment or Bedrock model serving eleven internal teams produces one cost line. There is no tag on a token.
So allocation has to be built where the context still exists: in the application. That means a request-level record — team, product, feature, customer, model, input tokens, output tokens, cached tokens — emitted by whatever calls the model, and joined to the bill afterwards. A gateway or proxy is the usual way to get this without changing every service. If you take one architectural decision from this page, make it this one: decide how AI spend will be attributed before the second team starts using the platform, because retrofitting attribution across a dozen services is significantly harder than adding it to one.
3. Cost and quality are coupled
You can nearly always buy a better answer. A larger model, more retrieved context, more reasoning effort, a second opinion from a second model — each improves output and each costs more. No classic cloud resource behaves this way: a right-sized VM does not produce worse answers, it produces the same answers more cheaply.
The consequence is that AI optimisation is not a search for the minimum. It is a search for the right point on a cost-quality frontier — which means no AI cost decision is safe to make without a quality measurement beside it. This is why cost per successful output matters so much more here than cost per unit does elsewhere: it is the only metric that prices the trade-off honestly, and it is why a cheap model can genuinely be the expensive option.
4. The spenders are not who you are used to
AI spend is driven by fast, sporadic experimentation, often by people who are not traditional engineers — data scientists, analysts, product managers with an API key. The Foundation's guidance is explicit that this changes the enablement problem, and many organisations respond with a cross-functional AI investment forum rather than routing decisions through existing engineering governance.
The practical implication for a FinOps team: your usual levers of code review and platform gatekeeping do not apply cleanly. Guardrails have to sit at the platform boundary — quotas, budgets, approved model lists, a gateway — because you cannot rely on reaching every spender through engineering process.
Mapping AI onto the FinOps Framework
You do not need a separate practice. You need your existing domains pointed at a new technology category. Here is the mapping, with the AI-specific work in each.
Understand Usage & Cost
- Data ingestion — get token counts and request counts per deployment alongside billed cost. All three major platforms publish these; most organisations simply are not reading them.
- Allocation — the request-level attribution described above. Without it, showback for AI is fiction.
- Reporting — report tokens and money. Tokens explain movements that money alone cannot, because a rate change and a volume change look identical on a cost chart.
- Anomaly management — classic anomaly detection under-performs on AI spend, because a model-mix shift can change cost dramatically at constant volume. You need detectors built for token spend: tokens per day, output-to-input ratio, blended rate per thousand tokens, and idle provisioned capacity.
Quantify Business Value
- Unit economics — cost per successful output, segmented by use case and, if you operate internationally, by language.
- Forecasting — forecast token demand, then price it, rather than extrapolating the dollar line. Token demand is driven by adoption and by prompt design, and those move independently.
- Budgeting — budget per use case rather than per service. "The Azure OpenAI budget" is not actionable; "the support-assistant budget" is.
- KPIs — see the metric set below.
Optimize Usage & Cost
- Usage optimisation — the token efficiency levers: right-size and route the model, trim the input, cap the output, cache the repeats, underpinned by the tokenization work.
- Rate optimisation — provisioned throughput and commitments, which pay only above a sustained utilisation threshold. Under-used provisioned capacity is one of the largest single AI line items we see, precisely because it bills flat regardless of traffic.
- Workload placement — batch versus interactive (batch endpoints are materially cheaper for anything not user-facing), region choice, and the prompt / RAG / fine-tune / self-host decision, which is really a choice between four different cost shapes.
- Architecture — for agentic systems, the controls that bound a loop. This is where the newest and largest exposure sits.
Manage the FinOps Practice
- Governance — approved models, quota per team, budget alerts, a required cost estimate before a workload goes live.
- Education — engineers changing a prompt are making a cost decision and usually do not know it. Token budgets per request shape make that visible in code review.
- Automation — cost checks in CI, scheduled assessments, alerting on unit cost rather than total spend.
- Cross-functional collaboration — the investment forum that decides which AI initiatives get funded and on what evidence.
The metric set
The Foundation's AI guidance anchors on token-normalised measures — cost per token, cost per inference, training cost efficiency — which are the right diagnostic layer because they are comparable across models and vendors. In practice you want three tiers:
- The decision metric — cost per successful output. One number, segmented by use case. This is what you optimise and what you report upward.
- The diagnostics — cost per inference, cost per token, average input tokens per request, average output tokens per request, prompt-cache hit rate, blended rate per thousand tokens, share of spend on premium models. These explain why the decision metric moved.
- The efficiency ratios — provisioned-throughput or commitment utilisation, failed-call rate (you pay for failures and then retry them), and for agents, cost per task at the median and the 99th percentile.
One discipline makes this set worth having: report the unit metric next to total spend, always. Total AI spend rising while cost per successful output falls is a business scaling well. Both rising is a problem. Only one chart tells you which you have.
A maturity ladder
Crawl — you can see it. AI spend is identified per service and per deployment. Token and request counts are pulled monthly. A named owner exists per deployment. Zombie deployments and idle provisioned capacity have been found and removed. This stage needs no code changes and typically returns the most money per hour invested.
Walk — you can attribute and optimise it. Request-level attribution exists, so showback by team and use case is real. Cost per successful output is defined for at least one significant workload. Caching, output caps and context trimming are in place on the highest-volume paths. Provisioned-throughput decisions are made on measured utilisation. Anomaly detectors run on token series, not just cost.
Run — you govern it. Every workload has a unit-cost target and a budget. Token budgets per request shape are enforced in CI. Agents ship with task budgets, step caps and timeouts as a matter of course. Model routing is measured against cost per successful output rather than assumed. Forecasts are built on token demand. AI cost is a standing item in the investment forum, with unit economics attached to funding decisions.
Most organisations we see are at the start of Crawl and believe they are at Walk, usually because a dashboard exists. The test is simple: can you say what your most-used AI feature costs per successful outcome? If not, you are at Crawl, and that is fine — Crawl is where the cheap money is.
A 90-day plan
Days 1–30 — measure. Inventory every AI deployment across every cloud and account, with billed cost and 30 days of token and request metrics. Kill the zombies. Check provisioned-throughput utilisation and downsize or move to pay-as-you-go where it does not pay. Pick the one workload that matters most and define its success bar.
Days 31–60 — attribute and trim. Add request-level attribution on the highest-spend path. Measure average input and output tokens per request. Prune tool schemas, tighten retrieval, cap output, enable caching with the stable prefix first. Compute cost per successful output for the chosen workload and set a baseline.
Days 61–90 — govern. Publish token budgets per request shape and enforce them in CI. Put task budgets, step caps and timeouts on any agent. Turn on token-series anomaly detection. Add the unit metric to the monthly business review next to total spend. Agree who approves a new model and who owns the quota.
The reference library
The guidance worth reading, grouped. These are primary sources — standards bodies and platform documentation — rather than commentary.
FinOps Foundation — framework and practice
- FinOps for AI — Framework technology category. The canonical definition, domains, personas and KPI guidance. Start here.
- FinOps for AI working group overview. Where the practitioner-led guidance is developed.
- The 2026 FinOps Framework update — technology categories, scopes and converging disciplines.
- State of FinOps data. The survey data behind claims about what practitioners actually prioritise; worth citing internally instead of vendor statistics.
- AI Value topic hub and AI for FinOps topic hub — the two directions are distinct and often conflated: managing the cost of AI, versus using AI to do FinOps.
- FinOps X 2026 keynote — token economics and the evolving role of FinOps.
Standards — FOCUS
- What is FOCUS? — the open billing-data specification, and our own plain-English explainer.
- The current FOCUS specification.
- FOCUS 1.2 announcement — SaaS/PaaS support and the virtual-currency columns that make token reporting standard.
Azure
- Well-Architected Framework — optimise component costs.
- Azure OpenAI prompt caching. Note the operational detail that catches people out: caching is automatic above a minimum prefix length, but the cache expires after a short idle period — so a low hit rate on a low-traffic deployment says nothing about how it is configured.
- Provisioned throughput (PTU) concepts — read before buying, and again before renewing.
AWS
- Well-Architected Generative AI Lens — GENCOST03-BP03, prompt caching to reduce token costs. The Lens has a full cost-optimisation set of best practices and is the most under-used AI cost resource AWS publishes.
- Well-Architected Agentic AI Lens — agent caching strategies. Relevant if you are running loops.
- Amazon Bedrock prompt caching — implicit and explicit caching, checkpoint minimums, and the detail that cache writes can bill above standard input.
- Our deep dive: Bedrock cost optimization.
Google Cloud
- Vertex AI context caching overview — implicit versus explicit caching, and the storage cost that explicit caching adds.
- Vertex AI generative AI pricing.
- Our deep dive: Vertex AI cost optimization.
Gateways and observability
If you need request-level attribution without modifying every service, a gateway is the usual answer. LiteLLM, Portkey and Helicone all sit in front of multiple providers and emit per-request token and cost records. Evaluate them on whether they capture cached-token counts separately — many do not, and cached input is where a large share of your savings will be.
Our deep dives, in reading order
- Token economics — how to meter, price and manage AI costs
- Tokenization — why the same meaning costs different amounts
- The token efficiency framework — four levers in ROI order
- Cost per successful output — the AI unit metric
- Agentic AI cost control — when one task costs forty calls
- Detecting AI cost anomalies
- AI cost governance — guardrails that hold
- Fine-tune, prompt, RAG or self-host — the cost decision
- Cloud unit economics — cost per customer
Getting the baseline without a project
Everything above assumes you know what you are spending, per deployment, today. Most teams do not — not because it is hidden, but because it is spread across metrics APIs and billing exports that nobody has joined up.
The CloudFinOpsKit FinOps Agent does that join for you. It reads 30 days of real token, request and failure metrics per deployment across Azure OpenAI and AI Foundry, Amazon Bedrock and SageMaker, and Vertex AI, prices each finding against your actual billed cost rather than list price, and reports the specific gaps this framework describes: zombie deployments, under-used provisioned throughput, cold prompt caches, context and output bloat, premium models carrying volume a smaller one could serve, and the failed-call overhead inflating the cost of every answer that worked. It runs read-only against your own subscriptions and accounts, so the Crawl stage of the ladder above is an afternoon rather than a quarter.
FAQ
What is FinOps for AI?
The application of FinOps practice to AI spend — now a technology category within the FinOps Framework. It covers allocation, forecasting, optimisation and governance for workloads billed in tokens and accelerated compute rather than resource hours.
How is AI spend different from normal cloud spend?
Four structural differences: the billing unit is a virtual currency you can influence; tokens carry no tags, so allocation must be built in the application; cost and quality are coupled, so optimisation targets a frontier rather than a minimum; and the spenders often sit outside traditional engineering governance.
What KPIs should I track?
Cost per successful output as the decision metric, with cost per inference, cost per token, average input and output tokens per request, cache hit rate and commitment utilisation as diagnostics. Add cost per task — median and 99th percentile — for agents.
Does FOCUS cover AI tokens?
Yes. FOCUS 1.2 added virtual-currency support via four pricing-currency columns, so token and credit consumption can be reported in the vendor's pricing currency and converted to a national one — which is what makes multi-vendor AI spend comparable in a single dataset.
Related reading: tokenization and cost · agentic AI cost control · token economics · cost per successful output · what is FOCUS? · the Cloud Center of Excellence