Towards connection-scalable RNIC architecture

Remote Direct Memory Access (RDMA) is a widely adopted optimization strategy in datacenter networking that surpasses traditional kernel-based TCP/IP networking through mechanisms such as kernel bypass and hardware offloading. However, RDMA also faces a scalability challenge with regard to connection management due to limited on-chip memory capacity in the RDMA Network Interface Card (RNIC). This necessitates the storage of connection context within RNIC's memory and induces considerable performance degradation when maintaining a large number of connections. In this paper, we propose a novel RNIC microarchitecture design that achieves peak performance and scales well with the number of connections. First, we model RNIC and identify two key factors that degrade performance when the number of connections grows large: head-of-line blocking when accessing the connection context and connection context dependency in transmission processing. To address the head-of-line blocking problem, we then combine a non-blocking connection requester and connection context management module to process prepared connections first, which achieves peak message rate when the number of connections grows large. Besides, to eliminate connection context dependency in RNIC, we deploy a latency-hiding connection context scheduling strategy, maintaining low latency when the number of connections increases. We implement and evaluate our design, demonstrating its successful maintenance of peak message rate (66.4 Mop/s) and low latency (3.89 µs) while scaling to over 50,000 connections with less on-chip memory footprint.