OpenRDMA: A GPU-Native Multi-Path Reliable RDMA Transport Architecture
Ziheng Ji, Chunyu Liu, Weihao Peng, Mingheng Mi, Pu Wang, Hai Zhang
As the computing power density of large-scale AI clusters rises, network communication increasingly limits system performance. GPU-initiated communication requires low-overhead request submission to overlap communication with computation. However, GPUDirect RDMA (GDR) still relies on CPU involvement in its control path, while InfiniBand GPUDirect Async (IBGDA) requires GPUs to assemble Work Queue Entries (WQEs), consuming additional SM resources and memory bandwidth. We propose OpenRDMA, an open-source multi-path reliable RDMA transport architecture designed for GPU-friendly communication. First, OpenRDMA provides a minimal GPU doorbell mechanism: the GPU issues an 8-byte token, and a dedicated FPGA NIC reconstructs the complete WQE from preconfigured context. Second, building on Multipath Reliable Connection (MRC), OpenRDMA supports per-packet multi-path transport and out-of-order arrivals, while selective retransmission, ordered DMA placement, and GPU-visible flag polling provide reliable delivery and safe data consumption. Third, a hardware-software co-designed retransmission mechanism keeps the GPU submission path lightweight and reduces FPGA hardware state overhead: the NIC handles fast data transfer, while CPU software manages complex timeout detection, state tracking, and recovery policies. We describe the architecture and key hardware components of OpenRDMA; implementation of the FPGA prototype is underway.
尚无 AI 解读，展示原文摘要