Loading page…
The preliminary comparison shifts attention from fixed prompt benchmarks toward the power required to keep long, tool-using AI sessions responsive.
Listen to this story
NVIDIA is positioning Vera Rubin as a major efficiency upgrade for agentic serving, but its headline remains an unverified vendor measurement rather than an independent benchmark result. In AgentX, a rack-scale Vera Rubin NVL72 configuration reportedly sustained up to 30x GB300 NVL72 throughput per megawatt at 160 output tokens per second per user.
AgentX measures long-context reuse, interactive decoding, tool-call gaps, dynamic concurrency and responsiveness—not just static token throughput.
NVIDIA reports GB300 delivered up to 15x H200 efficiency on DeepSeek V4 Pro and roughly 80x on Kimi K3.
The 30x figure covers a rack-level stack using SGLang, TensorRT-LLM, vLLM, Dynamo, optimized kernels, reduced-precision formats and NVLink.
NVIDIA says Vera Rubin NVL72 delivered up to 30 times the throughput per megawatt of GB300 NVL72 for an agentic AI workload at 160 output tokens per second per user. The result is preliminary: NVIDIA ran the AgentX measurement, and SemiAnalysis had not yet reviewed it when NVIDIA published the figure.
The comparison is aimed at AI agents that work through multi-turn coding sessions rather than a fixed prompt and response. AgentX replays production-style sessions and measures long-context prefill, KV-cache reuse, interactive decoding, tool-call gaps, dynamic concurrency and throughput per provisioned megawatt.
NVIDIA’s existing GB300 NVL72 comparisons show how widely results can move with the model under test. It reports up to 15 times higher throughput per megawatt than H200 NVL8 on DeepSeek V4 Pro 1.6T, and roughly 80 times higher on Kimi K3 2.8T at comparable interactivity. NVIDIA also puts GB300’s cost per million tokens at up to 10 times lower than H200’s in the cited DeepSeek comparison.
Throughput per megawatt alone does not establish whether an agent remains usable. AgentX pairs it with end-to-end normalized interactivity, generation-time interactivity, end-to-end latency and time to first token. Those measures capture both how much output a fixed power budget sustains and how quickly a session begins or finishes.
That workload focus distinguishes AgentX from a static sequence-length test. As sessions accumulate context and pause for tools or delegated work, a serving system must retain and reuse prior context while handling changing demand.
NVIDIA attributes GB300’s performance to its configured serving stack, including SGLang, TensorRT-LLM and vLLM; DeepGEMM-based kernels; MXFP4 and MXFP8 formats; Dynamo; and NVLink connectivity across 72 GPUs. NVIDIA says Dynamo can separately scale context processing and token generation, while its cache-aware routing weighs cache overlap and worker load.
That makes the 30x headline a claim about a particular workload, operating point and system configuration—not a standalone GPU measure. SemiAnalysis review remains the material test of NVIDIA’s preliminary Vera Rubin result.
Editorial analysis
The important contest here is no longer only accelerator speed. AgentX is designed around context growth, cache reuse, tool pauses and variable concurrency, which makes serving software and rack-level coordination central to the outcome. NVIDIA’s 30x figure is a sharp early claim, but the reviewed result is the event that matters next. It will show whether that advantage survives the benchmark’s evaluation process at the stated workload and interactive target. The telemetry work previously added to AgentX also offers a useful lens for judging how a system reaches a favorable curve, not just where it lands.
Loading discussion...
Be the first to share a perspective or an experience.
Reader comments
Newest comments first. Replies stay oldest first.