Lambda connects more than 10,000 GPUs with NVIDIA’s co-packaged optical switches
Lambda reports lower network latency without a throughput increase. NVIDIA’s larger efficiency and reliability claims describe the optics, not overall AI performance.
Loading page…
Lambda reports lower network latency without a throughput increase. NVIDIA’s larger efficiency and reliability claims describe the optics, not overall AI performance.
Listen to this story
Lambda’s deployment uses NVIDIA Photonics co-packaged optical switches to connect GB300 NVL72 racks containing more than 10,000 GPUs. The design moves optical engines beside the switch chip, shortening the electrical path and enabling separately replaceable external light sources. Lambda measured lower latency and cooler optics, while throughput matched conventional pluggable optics; NVIDIA’s larger power-efficiency and reliability figures are its own technology comparisons, not gains in whole-workload performance.
Each switch has 144 ports running at 800 gigabits per second, with liquid cooling for the switch chip and optical engines.
Lambda measured latency reductions of about 4% at rack scale and up to about 10% across a pod, with throughput unchanged.
Lambda reported median optics temperatures of 36°C, versus 51°C with conventional pluggable optics.
More than 10,000 GPUs in Lambda’s AI infrastructure are being connected through a different optical network design, targeting the power use and interruptions that can hold up large computing jobs. In a November 16 announcement, NVIDIA detailed Lambda’s deployment of its Photonics switches, which place optical engines beside the switch chip rather than in conventional plug-in modules.
Lambda’s tests found network latency—the time data takes to move—fell by about 4% at rack scale and by up to about 10% across a larger grouping called a pod. Throughput, or the amount of data carried, stayed the same as with conventional pluggable optics across rack, scalable-unit and pod configurations.
That distinction separates shorter delays from greater data-carrying capacity. The deployment connects NVIDIA GB300 NVL72 racks, where keeping processors working together depends on the network as well as the chips. NVIDIA notes that a single interrupted link during a large training job can stall thousands of GPUs; disruptions during AI-agent inference can spread delays across many agents.
The design is called co-packaged optics, or CPO. Optical engines convert electrical signals into light that carries data over fiber between parts of the cluster. Instead of putting that conversion in pluggable transceivers, NVIDIA places the engines alongside the switch chip on the same package. The electrical signal therefore travels a much shorter distance before becoming light.
Co-packaging cuts that electrical path from inches to millimeters, reducing signal loss. It also removes the need for retimer chips, which help signals along the electrical path. Removing them reduces both power consumption and the number of components in that path, according to NVIDIA. The change is in how the network moves signals, rather than in the GPUs doing the calculations.
Each switch in Lambda’s deployment has 144 ports running at 800 gigabits per second. Liquid cooling removes heat from both the switch chip and the optical engines. Lambda reports a median optics temperature of 36°C, against 51°C for conventional pluggable optics, and says its tests found more stable signal integrity.
Lambda reports a lower median optics temperature with the liquid-cooled Photonics switches than with conventional pluggable optics.
NVIDIA says Photonics delivers up to five times better power efficiency and up to 15 times higher reliability than conventional pluggable optics. Those are NVIDIA’s comparisons for the optical technology, distinct from Lambda’s latency and temperature results. They should not be read as equivalent gains in the speed or efficiency of an entire AI workload.
The company also says the design uses four times fewer lasers, reducing the number of components that can fail. For large clusters, its argument is that networking consumes electricity that could otherwise support computing, while a failed connection can interrupt work already underway. Reliability and power consumption are therefore central to the deployment’s stated purpose, alongside the measured reduction in delay.
Maintenance changes too. Each switch has 18 removable external light sources that supply the optical engines. Technicians can replace those light-source modules in the field. With conventional pluggable optics, a laser failure requires replacing the entire transceiver module; this design gives Lambda a separately replaceable light source.
Loading discussion...
Join the conversation
Explain which would matter more when choosing a provider.
Be the first to share a perspective or an experience.
Reader comments
Newest comments first. Replies stay oldest first.