Apple publishes four-Mac AI benchmark topping 36 tokens per second

Four linked Mac Studios can generate more than 36 tokens per second from Muse Spark Open, Apple reported in an October 15 technical post. Its new MLX benchmarks show how desktop computers can share an AI model too large for one machine, while revealing that faster output does not always mean a shorter initial wait.

First, make the model fit

The groundwork predates these results. In its June WWDC26 session, Apple demonstrated distributed inference across four M3 Ultra Macs. That walkthrough explained how MLX, its open-source machine-learning framework, uses Thunderbolt connections to coordinate machines. The October post adds measurements for M5 Ultra systems and a different set of models.

Muse Spark Open makes the memory problem concrete. It has 816 billion parameters—the numerical values that make up a model—with 42 billion active during inference. Apple tested a compressed, four-bit NVFP4 version occupying about 493GB. Although a Mac Studio supports up to 512GB of unified memory, the extra working memory needed to run the model pushes it beyond one machine’s capacity.

The smaller models provided single-machine comparisons. Qwen 3.8 has 27 billion parameters; Mistral Medium 3.5 has 128 billion. Apple evaluated them in their native BF16 and FP8 numerical formats, respectively. At the larger tested input size, they used up to 68GB and 158GB of memory, comfortably within a 512GB Studio.

Then, divide the work

The simplest approach splits the model’s layers between two computers. One processes the first half, then passes its output to the other. Called pipeline parallelization, this solves the capacity problem but generally leaves answer-generation speed close to a single machine’s: each token still travels through the layers in sequence. Long inputs can benefit because MLX overlaps the processing of separate chunks.

For more speed, tensor parallelization gives each computer part of every layer, allowing them to calculate together. The cost is frequent communication: machines must exchange inputs and combine outputs for every layer and token. Apple connects each pair of Studios with two Thunderbolt cables in a mesh, giving every machine a direct connection to every other.

Underneath is RDMA, or remote direct memory access, which moves data between machines’ memory while avoiding most processor and operating-system overhead. Apple’s stack combines macOS 26.2’s Thunderbolt 5 RDMA driver with JACCL, a communication library used by MLX version 0.30.1 and later.

Apple measured the delay before an answer starts, generation speed and peak power at input lengths of 8,192 and 65,536 tokens—chunks of text processed by the model. It used MLX LM’s benchmark tool; this example requests 128 output tokens from Qwen at batch size one:

mlx_lm.benchmark --model Qwen/Qwen3.8-27B -p 8192 -g 128 -b 1

Faster output, but not every stage

On a single Studio, Apple measured up to 4.6 times faster time to first token on M5 Ultra than M3 Ultra for the tested Qwen and Mistral workloads. It attributes that gain to specialized matrix-multiplication hardware. Generation was 1.6 times faster, with memory bandwidth rising from 819GB/s to 1.2TB/s.

Across machines, the result depends on which stage is running. At 65,536 input tokens, two-machine tensor parallelization processed Muse Spark Open’s prompt more slowly than the pipeline setup. Apple traces that to extra communication when combining the model’s expert outputs. Four machines improved the picture, while answer generation retained a substantial speedup even at the longer input size. The practical choice is therefore not simply how many Macs to connect, but how to divide the work.