On November 19, 2026, Microsoft announced a developer preview of "RDMA over USB4," which links compatible Windows PCs directly with USB4 cables and distributes large AI models, too big for a single PC's memory, across multiple machines. The matching Windows Insider build is coming soon to the Experimental Channel (Future Platforms), but the NetworkDirect provider that handles the communication and llama.cpp-based sample code are already available. In Intel's tests, an AI model that does not fit in the memory of a single 64 GB PC ran successfully on three PCs with the same specifications. However, when the setup was expanded to four PCs, generation speed improved while the time before output began got longer, showing that adding PCs does not necessarily improve every performance metric.

AD

Distributing an AI model too large for one PC over USB4

With RDMA over USB4, a compatible inference framework uses a technique called "tensor parallelism" to spread the computation of a single AI model across multiple PCs. Part of the model is placed in each PC's memory and computed on its own GPU, so even a large model that exceeds the memory capacity of one machine can be run. However, because the distributed results must be combined, the PCs exchange data many times even while generating a single token. Microsoft's llama.cpp validation fork includes an RPC backend that handles model splitting and uses NetworkDirect for communication between PCs.

Windows 11 already has a feature for connecting PCs to each other over USB4. Microsoft has now added a mechanism that lets RDMA (Remote Direct Memory Access) use the communication path that Windows' USB4 connection manager establishes between PCs. RDMA skips much of conventional network processing and transfers data directly between PCs' memory. The new provider handles sending and receiving through the USB4 host router's DMA rings, moving data between pre-registered memory regions without extra software copies.

NetworkDirect, the interface applications use, has long been used on Windows with dedicated RDMA adapters for Ethernet and InfiniBand. In this preview, simply connecting PCs directly with compatible USB4 ports and cables enables NetworkDirect communication. No dedicated RDMA adapters or network switches are needed. The published NetworkDirect v2 provider supports a model in which the sender and receiver each perform send and receive operations.

Microsoft measured round-trip latency for small messages between two PCs at roughly 28 to 50 µs, about one quarter that of Ethernet (UDP) communication over the same USB4 link. For a 16 KiB message, for example, RDMA over USB4 took 50 µs versus 194 µs for Ethernet over USB4. On 40 Gbps and 80 Gbps USB4 links, RDMA latency is similarly low, while the 80 Gbps link has twice the nominal bandwidth. According to Microsoft, 80 Gbps can be advantageous for communication-heavy workloads such as handling long inputs, but in tensor parallelism, which exchanges data frequently, the latency of each exchange also greatly affects performance.

In Microsoft's research facility, three Ryzen AI Max mini PCs, each with a Radeon 8060S and 128 GB of unified memory (memory shared by the CPU and GPU), were directly connected to one another over 40 Gbps USB4. Using AMD-optimized llama.cpp to run a 4-bit quantized version of the 235-billion-parameter mixture-of-experts (MoE) model Qwen3-235B-A22B, they recorded a generation speed of about 18.5 tokens per second.

AMD has also separately measured three Ryzen AI Max+ 395 mini PCs running a 4-bit quantized version (Q4_K_M) of the same model. With a context depth of 1,600 tokens and 2,048 generated tokens, generation speed after output began was 21.98 tokens per second, and input prompt processing speed was 117.98 tokens per second. The latter indicates how many input tokens can be processed per second and is different from the wait before a response begins. Intel measured that wait separately as the time until the first token is output.

A fourth PC raises generation speed but delays the first response

The "GPT-OSS-120B" model that Intel tested has about 120 billion parameters and does not fit in the memory of a single 64 GB PC of the type used. In the tests, three or four Intel Core Ultra Series 3 PCs were used, each with 64 GB of memory (transfer rate 9,600 MT/s). The PCs were connected directly through Thunderbolt 4 ports in a full-mesh configuration, with every PC linked to every other, and ran the same model.

Intel measured the time from input until the first token is output (Time to First Token, TTFT) and the generation speed after output begins (Decode Speed). In the results Microsoft published, performance with three and four PCs can be compared at context lengths of 1K and 2K.

Context length Time to first token (3 PCs → 4 PCs) Generation speed (3 PCs → 4 PCs)
1K 1.1 s → 1.2 s 49.4 → 54.2 tokens/s
2K 2.9 s → 3.2 s 49.4 → 51.7 tokens/s

Going from three to four PCs raised generation speed by about 9.7% at a 1K context length and about 4.7% at 2K. These increases were calculated by dividing the difference between the four-PC and three-PC speeds by the three-PC speed and multiplying by 100. Meanwhile, the wait for the first token grew by 0.1 seconds at 1K and 0.3 seconds at 2K.

The fourth PC added 4.8 tokens per second at 1K and 2.3 tokens per second at 2K. Although the time until the first output is longer, the longer the generated text, the more the later difference in generation speed affects total processing time. In other words, the benefit of adding PCs varies with input length and the number of tokens generated. Note that the table Intel published does not include the number of generated tokens, the number of measurement repetitions, the variation in results, or a breakdown of the time to first token.

AD

Directly connecting four PCs takes three ports per PC and six cables

Microsoft's basic requirement is two or more compatible PCs with USB4 ports managed by Windows, each connected directly to the others by USB4 cables. In a full-mesh configuration where all PCs can communicate directly with each other, three PCs need three cables and four PCs need six. With N PCs, each PC needs N−1 USB4 ports, and the total number of cables is given by the formula below.

[ \frac{N(N-1)}{2} ]

With two PCs, one port on each is used and a single cable connects them. Three PCs need two ports each, and four PCs need three ports each. Adding a fourth PC to an existing three-PC setup requires not only three more cables but also one free port on each existing PC.

In addition, a USB4 host router uses one pair of DMA rings, one for data and one for control, per RDMA link. The number of USB4 ports and DMA rings on a PC therefore limits the size of the cluster that can be built. Microsoft recommends connecting every PC directly to every other, noting that daisy-chaining PCs significantly degrades performance.

To use the developer preview, the compatible Windows Insider build, preview driver, and NetworkDirect provider version must match on all PCs. RDMA over USB4 is disabled by default, and enabling it requires registry settings and test signing. Because enabling test signing requires disabling Secure Boot, Microsoft recommends using dedicated test PCs rather than everyday machines. The release date of the compatible Insider build, the timing of the production release, and the supported models and final requirements at general availability have not been announced.

After connecting the PCs, assign a different subnet to each USB4 link and set static IPv4 addresses on the network interfaces at both ends. Microsoft provides a tool called "ndping" for checking connectivity on each link. Note that this preview has no automatic retry when communication is cut, so if a connection is lost, the application must reconnect. Specific setup instructions are published in the NetworkDirect provider's GitHub repository.

Microsoft's llama.cpp sample first establishes a connection over TCP and then switches to NetworkDirect communication. If a compatible provider is not installed or the adapter is unavailable, it continues over TCP, so you cannot tell whether RDMA is in use just because the program has started. Whether RDMA is actually enabled can be confirmed by a "NetworkDirect activated:" log line appearing for each link. Details of the behavior and how to check it are in the llama.cpp fork's RPC documentation.

The tests ran large AI models that cannot run on a single machine across multiple Windows PCs, and also confirmed a generation speed improvement with a four-PC setup. For real-world use, however, whether the necessary USB4 ports are available, along with how response time and generation speed change with input length and the amount of text generated, will inform the decision to add PCs.