FinOps for AI
Your token bill arrives as one number.
A model deployment costing thousands a month looks identical in your cost report whether it is serving a million useful requests or replaying a bloated prompt to nobody. Cost Management can tell you what you spent on AI. It cannot tell you whether that spend did any work. We read the token telemetry your platform already records, join it to your actual billed cost, and show you the difference.
Two directions, and they are not the same thing
The FinOps Foundation splits AI into two disciplines that get conflated constantly. We work in both, but they solve different problems and it is worth being precise about which is which.
Govern the cost of the AI you run
Token-level cost per model and per deployment across Azure OpenAI, AI Foundry, Bedrock, SageMaker and Vertex — priced from your real bill, not list price. This is the half with money in it, and the half almost nobody can see.
AI that amplifies the practitioner
The analyst in your report reasons over your findings and refuses to invent a number it cannot source. It amplifies expertise rather than replacing it — which is the only version of this that survives contact with a finance review.
What we actually measure
Thirty days of token counts, request counts and failure rates per deployment, joined to actual billed cost. No agent, no proxy, no SDK, no change to your AI applications — it reads what the platform already records. Prompt content is never accessed; only counts.
Zombie deployments
Models billing every month that served zero requests in 30 days. Entirely waste, and the cleanest money on the list — no quality trade-off, no code change.
Idle provisioned capacity
PTU and Bedrock Provisioned Throughput bill a flat hourly rate regardless of traffic. Because it looks like a fixed cost, nobody revisits it after purchase.
Cold prompt caches
The same large prefix re-sent uncached on every call, on a deployment busy enough that caching would work. Cached input bills at a fraction of standard input.
Output & context bloat
Unbounded replies and oversized context inflate every single call. Output bills several times higher than input, so an unset limit is an open cheque.
waste · necessary spend. Proportions above are illustrative of the shape of each problem, not measured averages — your report shows your real split, per deployment.
What a finding actually looks like
Every finding names the resource, quantifies the problem from measured telemetry, prices it against your billed cost, and tells you the next command to run. Here are two, in the report's own wording.
DetailDeployment gpt-4o-pilot (gpt-4o, Standard · capacity 60)
had ZERO requests over the last 30 days but the parent account still billed $412.80 last
month. Either an abandoned POC or a misrouted client.
RecommendationConfirm with the workload owner, then delete the deployment
(and the account if no other deployments share it).
az cognitiveservices account deployment delete -n <account> -g <rg> --deployment-name gpt-4o-pilot
Detail4,180,000 input tokens vs 214,000 output across 11,400 requests (cache match rate 3.1%). The same large prompt prefix is likely being re-sent uncached on every call.
RecommendationMark the stable prefix of the system prompt as cached. Put stable content first (frozen system prompt, deterministic tool list), volatile content (timestamps, varying user questions) after the cache breakpoint. Cached input typically bills at 10–25% of standard input.
These are example findings with illustrative figures — the wording, structure and thresholds are exactly what the engine emits, but the numbers are not a real customer's. We do not publish customer data. Two things worth noticing anyway: every saving is derived from actual billed cost rather than list price, and findings that depend on a judgement you have to make are labelled unverified rather than counted as banked savings.
What it does not do
The credibility of every number above depends on being straight about the edges:
- It does not measure output quality. The failed-call check is API-level success (HTTP 2xx). Whether an answer was usable is a judgement only your application can make.
- It does not know how hard your task is. Model right-sizing is flagged as a candidate for a quality test, never as a verdict.
- It cannot attribute tokens to a team. There is no tag on a token — one deployment serving ten teams is one cost line. That attribution has to be built in your application or a gateway.
- It never reads your prompts. Counts and status codes only.
The practitioner guides behind it
The checks above are the measurement half. These are the decisions they lead to — start with the framework if this is new.
- FinOps for AI: the complete framework The hub — domain mapping, KPI tiers, a maturity ladder, a 90-day plan, and the primary sources.
- Tokenization: why the same meaning costs different amounts The layer beneath every other lever — and why shortening your text can make it cost more.
- Token economics: metering, pricing and attributing AI spend How to meter at the call, price it in units the business understands, and attribute it.
- The token efficiency framework Four levers in ROI order: right-size the model, trim the input, cap the output, cache the repeats.
- Cost per successful output The unit metric that stops a cheap model quietly costing more than an expensive one.
- Agentic AI cost control Why doubling an agent's steps roughly quadruples its cost, and the controls that bound it.
- Detecting AI cost anomalies Detectors built for token spend, because classic anomaly detection misses a model-mix shift.
- AI cost governance Quotas, budgets, approved models — guardrails at the platform boundary.
- Fine-tune, prompt, RAG or self-host? Four options, four completely different cost shapes. Where the break-even actually sits.
Find out what your AI is actually costing
The assessment runs read-only against your own subscriptions and accounts, and the AI checks run as part of a normal scan — there is nothing extra to install.