AWS and Accenture say cost changes cut AI token spending by up to 85%
Four unnamed client engagements produced sharply different savings. The largest reductions combined several changes, limiting what they reveal about each technique.
Listen to this story
The audio brief
Story brief
3 key pointsAWS and Accenture’s November 19 guidance reports that four Amazon Bedrock engagements cut token spending by roughly 40% to 85%, but the results came from specific workloads and do not isolate the impact of individual changes. The strongest savings followed bundles of model routing, caching and workload adjustments; two other engagements used caching alone. For teams managing AI costs, the central lesson is to measure the full cost of successful outcomes and validate changes against production quality—not treat these case-study figures as portable forecasts.
- 01
In one enterprise deployment, combined model tiering, prompt caching and batching cut monthly token costs 85%; about 15% of tasks still needed the top model.
- 02
At a telecom operator, routing, semantic caching and context compression reduced input tokens 70%; annual spending fell from $12 million to $3.8 million.
- 03
An insurer and a mining operator used caching without routing or training; their roughly 40% savings are reported together, not separately.
Cutting AI spending did not require the same intervention at every client. In a November 19 publication, AWS and Accenture described four engagements with reported token savings ranging from about 40% to 85%. The biggest reductions combined several changes, making them evidence for an overall redesign—not a measured return on any single technique.
The findings cover a large enterprise deployment, a telecom operator, an insurer and a mining operator. Accenture withheld their names for client confidentiality. The figures are company-reported results specific to those workloads, not a promise that another organization using Amazon Bedrock would achieve the same savings.
Bigger savings came from bundled changes
The enterprise deployment started with every request going to a frontier model, without caching, model tiering or batching. Accenture introduced all three together: smaller models for suitable tasks, reuse of repeated prompt content, and grouped processing for overnight or non-urgent work. Monthly token costs fell 85%, but the publication does not divide that saving among the changes.
About 15% of tasks still required the top model for difficult, open-ended reasoning; lighter models handled the rest. That split describes this deployment, rather than a recommended allocation for every business.
The telecom operator had a different starting problem: agents generated requests without limits, and every request went to a frontier model regardless of complexity. Accenture combined triage routing, reuse of answers to similar questions, and compression of the context sent to models. It says input tokens fell 70%, and it validated output quality against the operator’s pre-change performance benchmarks.
Telecom operator’s annual token spending
Accenture reports a 68% reduction after applying routing, semantic caching and context compression together.
By contrast, the insurer and mining operator each added only one cache layer. The insurer reused static prompt prefixes; the miner returned stored answers to repeated, similar queries before they reached the model. The publication gives about 40% savings across those two engagements combined, without separate results. Neither involved routing changes or training investment.
Configuration first, training last
AWS and Accenture organize their six cost controls into three phases based on implementation complexity and investment. They recommend proceeding in order, while applying each phase independently where it fits a workload.
- Configuration changes: Prompt caching reuses computation for repeated instructions or documents. Batch inference groups work that can wait rather than requiring an immediate answer. The publication describes both as changes that need no new infrastructure.
- Architecture changes: Routing matches requests to suitable models. Retrieval optimization narrows the supporting material sent with a question. Accenture also scopes information to each agent’s task and compresses documents to reduce unnecessary input.
- Infrastructure and training: Semantic caching stores answers for reuse when questions have similar meanings. Distillation trains a smaller model using a larger model’s knowledge; the authors recommend it for stable, high-volume workloads that justify the training investment.
Count successful outcomes, including failed attempts
Before changing a workload, the authors recommend assigning spending to its owners through cost-allocation tags. Accenture’s Cost per Business Transaction framework then groups the model requests, retries, tool calls and agent handoffs needed for one successful business output. It includes failed attempts rather than treating the price of an individual model call as the complete cost of a result.
For teams applying the approach, the recommendation is more controlled than the bundled interventions behind the largest savings: establish cost and quality baselines, then change one lever at a time. Test quality on held-out examples from actual production traffic, not synthetic benchmarks, and retain a clean rollback path if quality falls below acceptable levels.
Sources
- aws.amazon.comOptimize AI economics on Amazon Bedrock with Accenture’s sequenced approach | Amazon Web Services
Reader comments
Newest comments first. Replies stay oldest first.