ArisGlobal says reordering AI prompts cut monthly inference spending by about 40%
A new AWS case study explains how shared instructions became more valuable to cache than lengthy pharmaceutical reports. The savings depend on repeated, identical input.
ArisGlobal’s NavaX pharmacovigilance workflow moved Amazon Bedrock’s cache checkpoint ahead of shared instructions, prioritizing reuse across cases and customers over reuse of each case document. AWS’s November 17 case study reports that cache reads cover 65% of input tokens and cut effective input costs about 50%; ArisGlobal says monthly inference spending fell roughly 40% at a volume exceeding 19 billion tokens. These are company-reported results, and every request still processes its uncached document, so savings depend on stable prompts and sufficient cache-hit traffic.
01
Over six months, shared instructions grew from about 72,000 to 180,000 tokens, while per-case inference costs rose more than 200%.
02
Bedrock caches matching opening prefixes, so changing earlier content can invalidate later reuse; changing customer and case data should follow the checkpoint.
03
ArisGlobal reports up to 25% lower end-to-end latency on cached production requests and about 15% more enterprise customers without proportional cost increases.
ArisGlobal found a cheaper way to run its pharmaceutical AI workflow by changing the order of what it sent the model. In a November 17 case study, AWS described a prompt-caching redesign that ArisGlobal says lowered monthly inference spending by approximately 40%. The change favored reusing shared instructions over reusing individual case documents.
The growing cost of repeated instructions
NavaX, ArisGlobal’s life-sciences AI engine on Amazon Bedrock, processes reports describing patients’ reactions to medications. Its extraction agent pulls around 130 fields from documents often spanning 10–200 pages, including patient details, suspect drugs, event timelines and regulatory classifications.
The model handles a case through several sequential requests, each focused on a different section of the form. Those requests carry the document alongside behavior rules, compliance constraints, field definitions and examples. Some instructions repeat across cases and customers using the same agent; the document repeats only within its own case.
As more pharmaceutical customers joined, their field mappings, jurisdiction rules and business requirements enlarged the instructions. Over six months, the shared opening portion of the prompt grew from roughly 72,000 to 180,000 tokens—the units used to measure model input. Per-case inference costs rose by more than 200%, according to the case study.
One boundary, a different order
Bedrock’s prompt cache reuses processing for an identical opening sequence, called a prefix. A checkpoint marks where that reusable sequence ends. Later matching requests can read the cached input at a reduced rate. It is not a collection of independent blocks: changing earlier content can invalidate matches for everything that follows.
ArisGlobal initially cached the source document because it was the largest part of a request. But each new case required a new cache entry, while the expanding instructions were charged at the full input rate every time. Moving those instructions ahead of the checkpoint made the more widely repeated material reusable instead.
A second checkpoint after the document could also cache the combined instructions-and-document prefix. That entry would still belong to one case, however. ArisGlobal chose one checkpoint after the shared instructions, giving up document reuse within a case to prioritize reuse across its customer base.
That approach needs consistent input and enough traffic to keep cache entries alive. For most active accounts, cases arrive about every five minutes per customer, with multiple customers running concurrently. Successful cache reads reset the expiry clock. AWS’s implementation guidance identifies three details that can determine whether reuse works:
Keep reusable instructions byte-for-byte identical; timestamps, request IDs and reordered tool definitions can cause misses.
Put customer identifiers, case data and the changing question after the cache checkpoint.
Check the model’s minimum prefix size and supported cache lifetimes rather than assuming one configuration fits every model.
The reported savings—and the remaining work
ArisGlobal reports that cache reads now supply 65% of input tokens in a typical request, lowering effective input costs by approximately 50%. At more than 19 billion tokens per month, it also reports up to 25% lower end-to-end latency on cached production requests and around 15% more enterprise customers without proportional cost increases.
These are ArisGlobal’s production metrics supplied to AWS, not independently verified results. AWS cautions that outcomes vary with prompt structure, traffic and cache-hit rates. The redesign left the system’s EU geographic cross-Region routing unchanged.
Each model call still processes the uncached case document. AWS therefore recommends combining sequential extraction questions when output limits and schema requirements allow—reducing the work that remains after the cache has done its part.
Sources
aws.amazon.comHow ArisGlobal scaled generative AI inference on Amazon Bedrock | Amazon Web Services
Reader comments
Newest comments first. Replies stay oldest first.