Meta unveils MetaRoCE: a new rdma protocol optimized for AI workloads

Meta AI has introduced MetaRoCE, a novel RDMA transport protocol designed specifically for AI-scale Ethernet networks. The protocol represents a significant departure from traditional RoCE by treating the network as lossy and shifting key responsibilities such as ordering, path selection, and recovery to the network interface card (NIC). This approach allows for more efficient handling of collective operations like all-reduce and all-to-all, which are critical in training large-scale AI models. MetaRoCE is being developed as part of the Open Compute Project (OCP), with the specification, a reference implementation, and a compliance test suite set to be released at the 2026 OCP Global Summit. The protocol has already been tested on AMD Pensando programmable NICs, with additional implementations in progress from other vendors. Hardware support is currently in early stages, and the focus remains on fabric-architecture decisions rather than procurement. The protocol's performance has been validated on a 64-node AMD GPU cluster, where it outperformed RoCEv2 in both throughput and flow completion times. MetaRoCE maintains approximately 86% throughput even at 1% packet loss and continues to deliver useful bandwidth at 10% loss rates, demonstrating its resilience. Multiplane validation across 4-plane and 8-plane topologies confirmed linear scalability with plane count, and simulated plane failures showed traffic redistribution without application or operator intervention. MetaRoCE extends the multi-vendor philosophy established by OCP’s Ethernet Scalable Unified Network (ESUN) initiative into the transport layer. Three key artifacts are being released: the full specification, a compliance suite for vendor validation, and libsoftmetaroce as the authoritative behavioral model for silicon development. The protocol builds on Meta’s 2024 RoCE-at-scale work and broader infrastructure evolution, marking a significant step in optimizing network performance for AI workloads.