Microsoft previews linking Windows PCs over USB4 to run larger AI models
Open-source components are available now, but the required Windows Insider build is still coming. Early company benchmarks show both the capacity gains and the limits of adding another PC.
Microsoft’s Nov. 19 developer preview adds RDMA over USB4 to Windows’ NetworkDirect interface, letting compatible frameworks use multiple PCs’ combined memory and graphics to run models too large for one machine—without a dedicated RDMA adapter or network switch. The open-source provider and llama.cpp demonstration are available, but the required Windows Insider build and setup instructions have not arrived, making this a developer starting point rather than a plug-and-play upgrade. Early tests show potential, though performance depends on the model and configuration.
01
Microsoft measured 28–50 microseconds of small-message round-trip latency and roughly four times less latency than Ethernet UDP over the same USB4 link.
02
Microsoft’s three-PC AMD test ran Qwen3-235B-A22B at about 18.5 tokens per second; AMD reported 21.98 tokens per second on its own setup.
03
Intel’s GPT-OSS-120B test reached 49.4 tokens per second on three PCs and 54.2 on four, while time to first token rose from 1.1 to 1.2 seconds.
Windows PCs connected by USB4 cables will be able to share the work of running an AI model too large for one machine. Microsoftannounced the developer preview on November 19, with open-source components available now. The required Windows Insider build is still coming, making this a development starting point rather than a ready-to-use desktop feature.
The capability, called RDMA over USB4, lets a compatible AI framework split one model across two or more PCs and use their combined memory and graphics processing. It does not require a dedicated RDMA network adapter or a network switch. Microsoft has published a modified version of llama.cpp, software for running AI models, to demonstrate the approach.
A shorter path between machines
RDMA stands for Remote Direct Memory Access. It moves data between computers’ memory while bypassing much of the usual networking software. Windows already supports it through NetworkDirect, a programming interface used with specialized data-center network adapters. The preview brings that interface to USB4 connections between PCs.
That matters when a framework divides a model’s calculations among machines using tensor parallelism: the PCs must exchange data repeatedly for every token, or piece of generated text. Microsoft says communication delay can then set the pace more than raw bandwidth. Its USB4 implementation transfers data between registered memory regions without extra software copies.
USB4 communication latency
28–50 microsecondsSmall-message round-trip delay
Microsoft measured roughly four times lower latency with RDMA than with Ethernet using UDP over the same USB4 link.
Larger models, uneven scaling
Microsoft’s lab ran Qwen3-235B-A22B, a 235-billion-parameter model, across three AMD Ryzen AI Max mini-PCs with 128GB of shared CPU and GPU memory each. With four-bit quantization—storing model weights at lower numerical precision—it generated about 18.5 tokens per second. AMD reported 21.98 tokens per second on its own three-PC setup.
Intel tested GPT-OSS-120B on Core Ultra Series 3 PCs with 64GB of memory each, a capacity Microsoft says cannot hold that model alone. At a 1,000-token context, three machines generated 49.4 tokens per second; four reached 54.2. The fourth PC did not shorten every wait: time to the first token rose from 1.1 to 1.2 seconds.
These are company-supplied measurements, not a uniform comparison between AMD and Intel. Microsoft says performance depends on the model, framework build, memory and GPU. It has tested up to three AMD PCs and four Intel PCs, while describing those systems as tested platforms rather than the only compatible hardware.
A test cluster, not a cable-and-go upgrade
Microsoft’s setup requirements make clear how much work remains for the developer:
Connect every PC directly to every other PC. Three machines require two USB4 ports per machine and a cable between each pair. Microsoft warns that daisy-chaining sharply reduces performance.
Use the same supported Insider build, preview drivers and NetworkDirect provider version on every PC. Each USB4 network connection also needs a static IPv4 address on its own subnet.
Enable test signing, which requires turning off Secure Boot. Microsoft recommends dedicated test PCs rather than an everyday device, and instructs testers to restore Secure Boot afterward.
Applications also have to handle recovery: the transport does not retry a dropped NetworkDirect connection. The application must re-establish it.
For now, developers can prepare applications using the published provider and sample code. Microsoft says it will add the required build and update instructions when the preview reaches the Windows Insider Experimental channel’s Future Platforms track.
Sources
techcommunity.microsoft.comIntroducing RDMA over USB4® on Windows: run larger local AI models across PCs | Microsoft Community Hub
Reader comments
Newest comments first. Replies stay oldest first.