Loading page…
The company’s benchmark isolates CPU work from network calls. Adopting Vera would require changes to Modal’s historically x86-only infrastructure.
Listen to this story
Modal's October 19 benchmark found Nvidia's Vera CPU completed roughly twice as many tasks as competing processors when many sandbox jobs ran concurrently, though it showed no consistent, large lead on single-task runs. The result is an internal evaluation, not a deployment decision or customer rollout. Vera could help Modal share burst capacity and reduce CPU-side waits in synchronous reinforcement-learning runs, but adopting it would require the historically x86-only company to add Arm support across its infrastructure and image-building system.
Modal's Agentless Agent Bench ran 50 Terminal-Bench reference solutions and their checkers in prepared Docker containers, with network calls removed.
The throughput result measures total tasks completed in a fixed period; it does not mean an individual agent job runs twice as fast.
Modal reserves capacity around typical sandbox usage and allows bursts above it, relying on spare host capacity and accepting more risk of customer interference.
Modal says Nvidia’s Vera CPU delivered roughly twice the throughput in its test of concurrent sandbox workloads, strengthening its case for changing the processors behind Modal Sandboxes. The October 19 benchmark publication describes an evaluation of a potential processor for the service, not a customer rollout.
Nvidia hand-delivered Modal’s first Vera CPU server to its San Francisco office the week before publication, according to author Erik Dunteman. Vera is an Arm-compatible processor built for Nvidia’s Vera Rubin GPU platform, but it is also available in a standalone CPU server.
The company built Agentless Agent Bench to simulate the varied CPU work performed inside its sandboxes. It uses 50 Terminal-Bench reference solutions and their result-checking programs, with network calls removed. Tasks run in concurrent Docker containers prepared in advance.
The workload mixes compilation, file searches, Git operations and device emulation. Its first experiment ran tasks one at a time on an otherwise unloaded machine. Vera often finished faster than the other CPUs, but not consistently and not by a large margin.
Modal then increased the number of containers running together and counted how many tasks finished within a fixed period. That measures throughput: total completed work, rather than the time one task takes. The stronger Vera advantage appeared under this competing load. These are Modal’s own measurements, not an independent evaluation.
That crowded setting is central to Modal’s scheduling problem. Hundreds of containers and virtual machines can share a host. A sandbox may sit mostly idle while waiting for a language model, then suddenly demand much more CPU capacity. One customer’s burst can slow another customer’s work.
Modal Sandboxes lets customers reserve guaranteed capacity around typical usage and temporarily burst above it, rather than reserve for their largest possible spike. The approach relies on spare host capacity. Modal says this increases exposure to interference between customers compared with strict limits.
CPU delays can also hold up more expensive equipment. Modal points to reinforcement-learning training, where sandboxed processes carry out trial runs. In synchronous setups, the GPU system waits until an entire group of those runs finishes before advancing. Shortening CPU work can therefore reduce GPU waiting time.
For Modal, the next step involves more than installing hardware. The company has historically run exclusively on x86 machines. Supporting Arm would require changes across its infrastructure and its system for building the software images that workloads run from.
Loading discussion...
Join the conversation
Explain when flexibility would outweigh the risk of another workload slowing yours.
Be the first to share a perspective or an experience.
Reader comments
Newest comments first. Replies stay oldest first.