Meta Sends MetaRoCE to OCP for Loss-Tolerant AI Ethernet

Meta has designed MetaRoCE, a clean-sheet RDMA transport for AI workloads on commodity Ethernet, and is contributing its specification, reference software and compliance suite through the Open Compute Project. The protocol is built to accept packets arriving out of order and to recover from loss without asking Ethernet switches to pause traffic. That addresses a costly AI-cluster problem: during synchronized training work, the slowest transfer can set the job’s pace.

Packets take many routes, and the NIC puts them back to work

Meta says standard RoCE expects frames to arrive in order, uses priority flow control, or PFC, and discourages packet spraying across routes in large multiplane networks. MetaRoCE instead sprays packets across paths and treats out-of-order arrival as its normal operating condition.

Each packet carries destination information, allowing the receiving NIC to write data directly to its final memory location. Meta says this removes the need for a reorder buffer and avoids head-of-line blocking, where a delayed packet holds up later arrivals. The design also does not require PFC or pause frames.

Path-level recovery replaces a connection-wide stall

A MetaRoCE connection has distinct paths, each with its own round-trip measurement, congestion state, utilization data and congestion window. The NIC uses different UDP source ports to give equal-cost multipath routing distinct traffic to distribute, then can shift traffic away from a troubled route.

Selective acknowledgments identify a missing packet on the affected path and trigger its retransmission there. MetaRoCE pairs sender-side ECN congestion feedback with receiver-provided rate hints, so congestion can slow the marked path while traffic moves toward clearer ones.

~86%Throughput retained

Meta reports that MetaRoCE retained about 86% throughput at 1% packet loss and continued delivering useful bandwidth at 10% loss in its tests.

The open specification now needs more hardware

MetaRoCE separates ordered message streams from bandwidth control, allowing one connection to carry many streams and many paths under one congestion controller. Meta says existing RDMA Verbs APIs and software stacks work without modification, while extension APIs expose features such as multiplane support.

Meta and AMD implemented the transport on AMD Pensando programmable NICs and evaluated it on a 64-node AMD GPU cluster running RCCL collectives. Meta says it beat RoCEv2 on throughput and flow-completion time in those tests. It plans to release a DPDK-optimized reference implementation, specification and compliance framework at the OCP Global Summit in October 2026; additional vendor implementations are under way.