LLM spend has a strange property: list prices fall relentlessly — OpenAI cites a 97% price-per-token decline from GPT-4 to its current flagship — yet enterprise AI bills keep rising, because usage grows faster than prices fall. The teams that control spend treat inference like cloud cost management circa 2015: an engineering discipline with owners, dashboards, and targets. Here are the nine tactics that matter, ordered roughly by effort-to-impact.
1. Route by Task Difficulty (30–70% savings)
The single biggest lever. Most production traffic is easy — classification, extraction, short answers — and does not need a frontier model. A router (rules-based is fine to start) that sends easy traffic to a small model and hard traffic to a frontier model typically cuts spend by half. DeepSeek's V4-Flash lists at $0.14 per million input tokens; frontier models cost 10–40× that. The quality bar is your eval set, not the leaderboard.
2. Prompt Caching (up to 90% on cached input)
If your prompts share a long common prefix — system instructions, tool definitions, reference documents — cached-input pricing turns those tokens nearly free. Restructure prompts so the static part comes first. For agent workloads with big tool schemas, caching is often the difference between viable and not.
3. Batch APIs (typically 50% off)
Anything that doesn't need a real-time answer — nightly enrichment, backfills, evals — belongs on batch endpoints at roughly half price. The main cost is architectural: making pipelines async.
4. Trim the Context, Not the Quality
Token bills are usually dominated by input, not output. Common waste: entire documents where a retrieved passage would do, unbounded chat history, verbose tool outputs echoed back. Context discipline routinely cuts input tokens 40%+.
5. Open-Weight Models for Commodity Workloads
For high-volume, stable tasks (moderation, embedding, routing itself), self-hosted or hosted open-weight models undercut API pricing at scale — and per-second billed endpoints (e.g., Hugging Face's one-command OpenAI-compatible endpoints) remove the fixed-cost barrier to trying.
6. Right-Size the Reasoning
Reasoning modes bill their thinking tokens. Configurable reasoning levels (now standard across providers) mean you can dial effort per task class. Full reasoning on a formatting task is pure waste.
7. Use the Contract Levers
At enterprise volume, list price is the starting point: committed-use discounts, dedicated capacity (per hour-instance pricing rather than per token), and introductory windows on new models all move the number. Watch launch pricing — models sometimes debut cheap and step up on a date; time migrations accordingly.
8. Kill Zombie Usage
Dashboards reveal it every time: dev loops hammering production keys, retries multiplying failures, forgotten cron jobs summarizing nothing anyone reads. Tag every caller, review monthly, and delete.
9. Re-Baseline Quarterly
Providers reprice constantly and retire models on hard deadlines. A quarterly review that re-runs your eval set against the current price/quality frontier — and re-routes traffic accordingly — compounds all eight tactics above. Assign it an owner.
The Operating Model
Put one person in charge of unit economics (cost per resolved task, not cost per token), give every use case a budget, and alert on deviation. Teams with this discipline routinely run the same workload portfolio at a third of the cost of teams without it.
Internal Link Suggestions
How to Choose an LLM Vendor: A CIO's Guide → aitj-llm-vendor-selection-cio-guide
How to Measure the ROI of AI Agents → aitj-how-to-measure-ai-agent-roi
Open vs. Closed AI Models in 2026 → aifn-open-vs-closed-models-2026
