AWS says prompt caching cut AI input costs by 62% in a six-turn test

Re-reading an agent’s past does not have to mean paying full input rates every turn. In AWS’s October 26 guide, the company reported about 62% lower input costs for GPT-6 Astra on Amazon Bedrock using automatic prompt caching in a six-turn workload. Explicit rolling breakpoints delivered about 61% savings against the same no-caching baseline.

Reuse the computation, not the answer

An agent that calls software tools repeatedly sends the model its instructions, tool definitions and conversation history, followed by the newest message or tool result. The model has no internal memory between requests. As the task continues, earlier steps remain in the input, so the amount of repeated material grows alongside the conversation.

Prompt caching saves work done during prefill: the processing that happens before the model produces its first output token. The service saves the model’s working state for the beginning of a prompt. When a later request starts with identical input, it reloads that state and processes only the new material.

That reusable beginning, called a prefix, must contain at least 1,024 tokens—units of model input—to qualify in the guide’s setup. Its examples configure a 30-minute cache lifetime. Setting store=False leaves conversation history in the application; it does not disable prompt caching.

One boundary for shared rules, another for growing history

Implicit caching lets Bedrock choose where the reusable prefix ends. Explicit caching puts that decision in the developer’s code through a prompt_cache_breakpoint marker. AWS recommends combining them: mark the shared tools and instructions explicitly, while allowing automatic caching to follow the conversation as messages and tool results accumulate.

The distinction becomes important across different tasks. An automatic cache entry ending after one user’s task may not match a different task, even when both share the same instructions. An explicit boundary before the changing task preserves that shorter starting point for reuse across sessions or by helper agents.

For explicit-only caching, AWS also demonstrates rolling boundaries: mark the last tool result in each batch and retain earlier markers. That allows each turn to reuse prior history rather than repeatedly processing everything after a single fixed boundary.

AWS’s six-turn GPT-6 Astra comparison

01About 62% lower input cost

Implicit caching

AWS measured this reduction against running the six-turn workload without caching.

02About 61% lower input cost

Explicit rolling breakpoints

AWS measured this reduction on the same model and workload against the no-caching baseline.

The savings depend on what stays unchanged

AWS cautions that these are workload-specific results and tells developers to measure their own agents. Its guide identifies several ways an apparently reasonable setup can lose reuse:

• Changing the prefix. Instructions and tool definitions must serialize identically. Put variable details, such as dates, user names and request IDs, after the cached instructions.
• Rebuilding the tool catalog. Adding, removing or reordering initial tool definitions can invalidate the prefix. Keep the initial list stable.
• Caching only the static setup. In explicit-only mode, one fixed boundary saves tools and instructions but leaves the growing history to be processed again each turn.

Caching also has a write cost. In its July 30 explanation for the earlier GPT-5.6 models, AWS listed cache reads at a 90% discount to ordinary input and writes at 1.25 times the uncached rate. It estimated that net input costs fall when reads reach roughly 20% of tokens flowing through the cache. Those figures describe GPT-5.6 pricing, not a rate card for the newer Astra test.

The practical check is in each response’s usage data. AWS recommends logging cached_tokens and cache_write_tokens alongside the request ID and cache key. A working write-once, read-many pattern should show an initial cache write, followed by reads of the reusable prefix on later requests.