Cerebras pairs GPUs with its chips in production, claiming fivefold output per megawatt
The company’s production comparison uses AMD GPUs alongside CS-3 systems, measured against CS-3 alone. Broader deployment is planned through 2027.
Loading page…
The company’s production comparison uses AMD GPUs alongside CS-3 systems, measured against CS-3 alone. Broader deployment is planned through 2027.
Listen to this story
Cerebras’s production setup separates inference stages: AMD Instinct GPUs handle prompt prefill, while CS-3 systems generate responses. On a shared GPT-OSS-120B endpoint, Cerebras reported five times the output per megawatt versus a CS-3-only baseline, with no slowdown in token generation. The comparison was conducted on November 14 and is limited to one endpoint; it does not establish that other models or deployments will see the same gain.
KV Portal transfers intermediate state, including key-value caches, between the prefill and decode engines using RDMA.
Infrastructure partners can choose Cerebras’s end-to-end stack or a decode-only SDK that leaves them more control over their serving framework.
Amazon Web Services, Fireworks AI, Gimlet Labs, and General Compute have early access; Cerebras is opening the architecture to additional infrastructure partners.
Cerebras says a new division of labor between chips delivered five times as much AI output per megawatt without slowing token generation. In its November 17, 2026 announcement, the company said its disaggregated inference system is in production: GPUs process incoming prompts, while Cerebras systems concentrate on generating the responses.
The production deployment runs on Cerebras Inference Cloud, pairing AMD Instinct GPUs with Cerebras CS-3 systems. Moving prompt processing off the CS-3 systems lets them devote their serving capacity to producing answers, rather than dividing that capacity between reading requests and writing responses.
Cerebras measured the change on November 14 using a shared GPT-OSS-120B endpoint, comparing throughput before and after switching on the split setup. The baseline was a CS-3-only configuration. That makes this a company-reported production comparison for one endpoint—not a general claim that the same gain applies to every model or deployment.
Inference—the work of running a model to produce a response—has two main stages. Prompt processing, called prefill, reads the input and creates intermediate model state. Response generation, called decode, uses that state to produce output tokens, the pieces of text that make up an answer.
Decode generates one token at a time, feeding each new token into the next step. Those repeated steps require repeated reads of model data, making memory bandwidth important. Cerebras’s argument for handling this stage rests on keeping data in fast on-chip memory, known as SRAM, close to the computing hardware.
The split also creates a handoff between the two stages. Cerebras supplies KV Portal software to move intermediate model state, including key-value caches, from the prompt-processing engine to the response-generation engine. It uses RDMA, a method for transferring data directly between machines’ memory, to carry that state across the network.
Infrastructure partners can build clusters using a Cerebras reference architecture, placing their chosen prompt-processing accelerators and Cerebras systems on a unified network. The partner integration options offer two routes, depending on whether operators want a ready-made software stack or control over more of the serving system.
Cerebras says Amazon Web Services, Fireworks AI, Gimlet Labs and General Compute already have early access to the solution for their deployments. It is now opening access to additional infrastructure partners that want to use the split architecture in their own data centers.
The next announced milestone is broader deployment across hundreds of megawatts in Cerebras Inference Cloud and partner installations by the end of 2027. That remains a rollout plan, beyond the production deployment Cerebras has described.
Loading discussion...
Join the conversation
Explain where you would draw the line between convenience and control.
Be the first to share a perspective or an experience.
Reader comments
Newest comments first. Replies stay oldest first.