Publications
* denotes equal contribution.
2027
- NSDIPipeTune: An Efficient Performance Tuning Framework for Host DatapathsXinyang Huang, Zhuobin Huang , Qijing Li, Junxue Zhang, Mengyu Ma, Han Tian, Bowen Liu, and Kai ChenIn USENIX Symposium on Networked Systems Design and Implementation (NSDI’27) , 2027
The host datapath is undergoing a revolutionary transformation, wherein the bottleneck has shifted from CPU to memory subsystem, accompanied by advanced CPU offloads and memory techniques. However, existing datapaths suffer from significant performance degradation when adapting to these changes. To this end, we identify 6 tunable datapath configurations that are crucial for high performance by investigating 9 datapath implementations, yet conventional wisdom for tuning yields non-monotonic effects and inconsistent results across deployments. This paper presents PipeTune, an efficient tuning framework that addresses performance unpredictability and deployment inconsistency. At its core, PipeTune systematically models the correlation among configurations, memory efficiency, and end-to-end datapath performance, establishing 4 effective guidelines for configuration adjustment. Building upon these insights, PipeTune automatically tunes identified configurations alongside the original datapaths during bootstrap, while running as a lightweight daemon for runtime diagnosis and optimization. We have fully implemented PipeTune and evaluated it on five widely used datapaths. With PipeTune, these datapaths successfully deliver near line-rate throughput and microsecond-scale tail latency across diverse hardware and workloads. Compared with current practice, PipeTune improves the throughput and reduces the P99.9 latency of target datapaths by up to 2.1x and 4.6x, respectively.
2025
- SIGCOMMMixNet: A Runtime Reconfigurable Optical-Electrical Fabric for Distributed Mixture-of-Experts TrainingXudong Liao, Yijun Sun, Han Tian, Xinchen Wan, Yilun Jin , Zilong Wang, Zhenghang Ren , Xinyang Huang, Wenxue Li, Kin Fai Tse, Zhizhen Zhong , Guyue Liu , Ying Zhang, Xiaofeng Ye , Yiming Zhang, and Kai ChenIn 39th ACM Special Interest Group on Data Communication (SIGCOMM’25) , 2025
Mixture-of-Expert (MoE) models outperform conventional models by selectively activating different subnets, named experts, on a per-token basis. This gated computation generates dynamic communications that cannot be determined beforehand, challenging the existing GPU interconnects that remain static during distributed training. In this paper, we advocate for a first-of-its-kind system, called MixNet, that unlocks topology reconfiguration during distributed MoE training. Towards this vision, we first perform a production measurement study and show that the MoE dynamic communication pattern has strong locality, alleviating the need for global reconfiguration. Based on this, we design and implement a regionally reconfigurable high-bandwidth domain that augments existing electrical interconnects using optical circuit switching (OCS), achieving scalability while maintaining rapid adaptability. We build a fully functional MixNet prototype with commodity hardware and a customized collective communication runtime. Our prototype trains state-of-the-art MoE models with in-training topology reconfiguration across 32 A100 GPUs. Large-scale packet-level simulations show that MixNet achieves performance comparable to a non-blocking fat-tree fabric while boosting the networking cost efficiency (e.g., performance per dollar) of four representative MoE models by 1.2×–1.5× and 1.9×–2.3× at 100 Gbps and 400 Gbps link bandwidths, respectively.
- SIGCOMMCEIO: A Cache-Efficient Network I/O Architecture for NIC-CPU Data PathsBowen Liu* , Xinyang Huang* , Qijing Li, Zhuobin Huang, Yijun Sun, Wenxue Li, Junxue Zhang, Ping Yin, and Kai ChenIn 39th ACM Special Interest Group on Data Communication (SIGCOMM’25) , 2025
Efficient Input/Output (I/O) data path between NICs and CPUs/DRAMs is critical for supporting datacenter applications with high-performance network transmission, especially as link speed scales to 100Gbps and beyond. Traditional I/O acceleration strategies, such as Data Direct I/O (DDIO) and Remote Direct Memory Access (RDMA), perform suboptimally due to the inefficient utilization of the Last-Level Cache (LLC). This paper presents CEIO, a novel cache-efficient network I/O architecture that employs proactive rate control and elastic buffering to achieve zero LLC misses in the I/O data path while ensuring the effectiveness of DDIO and RDMA under various network conditions. We have implemented CEIO on commodity SmartNICs and incorporated it into widely-used DPDK and RDMA libraries. Experiments with well-optimized RPC framework and distributed file system under realistic workloads demonstrate that CEIO achieves up to 2.9x higher throughput and 1.9x lower P99.9 latency over prior work.
- SIGCOMMRevisiting RDMA Reliability for Lossy FabricsWenxue Li , Xiangzhou Liu , Yunxuan Zhang , Zihao Wang, Wei Gu , Gaoxiong Zeng , Shoushou Ren , Xinyang Huang, Zhenghang Ren, Bowen Liu, Junxue Zhang, and Kai ChenIn 39th ACM Special Interest Group on Data Communication (SIGCOMM’25) , 2025
Best Student Paper
Due to the high operational complexity and limited deployment scale of lossless RDMA networks, the community has been exploring efficient RDMA communication over lossy fabrics. State-of-the-art (SOTA) lossy RDMA solutions implement a simplified selective repeat mechanism in RDMA NICs (RNICs) to enhance loss recovery efficiency. However, these solutions still face performance challenges, such as unavoidable ECMP hash collisions and excessive retransmission timeouts (RTOs). In this paper, we revisit RDMA reliability with the goals of being independent of PFC, compatible with packet-level load balancing, free from RTO, and friendly to hardware offloading. To this end, we propose DCP, a transport architecture that co-designs both the switch and RNICs, fully meeting the design goals. At its core, DCP-Switch introduces a simple yet effective lossless control plane, which is leveraged by DCP-RNIC to enhance reliability support for high-speed lossy fabrics, primarily including header-only-based retransmission and bitmap-free packet tracking. We prototype DCP-Switch using P4 switch and DCP-RNIC using FPGA. Extensive experiments demonstrate that DCP achieves 1.6× and 2.1× performance improvements, compared to SOTA lossless and lossy RDMA solutions, respectively.
- arXivSwift: Rethinking RDMA Control Plane for Elastic ComputingarXiv preprint arXiv:2501.19051, 2025
- APNetCache-Aware I/O Rate Control for RDMAIn 9th Asia-Pacific Workshop on Networking (APNet’25) , 2025
Remote Direct Memory Access (RDMA) has become a cornerstone technology in modern datacenter networks due to its high throughput and extremely low latency. However, recent works have revealed that congestion arises in the "last mile" of the RDMA I/O path—–between DRAM and CPU registers–—due to inefficiencies in the memory hierarchy, where severe cache misses and memory bandwidth contention degrade performance. We identify the root cause of this I/O congestion as the speed mismatch between network ingress and CPU processing, which leads to data accumulation and, eventually, Last-Level Cache (LLC) overflow. To address this issue, we propose RhyR, a credit-based rate control mechanism that dynamically aligns network ingress speed with CPU processing speed. Our preliminary evaluation on eRPC over RDMA, a widely used RPC framework, demonstrates that RhyR effectively mitigates I/O congestion, reducing tail latency by up to 1.40x and improving throughput by up to 1.35x compared to prior work.
- OSDIEnabling Efficient GPU Communication over Multiple NICs with FuseLinkZhenghang Ren , Yuxuan Li , Zilong Wang , Xinyang Huang, Wenxue Li, Kaiqiang Xu, Xudong Liao, Yijun Sun, Bowen Liu, Han Tian, Junxue Zhang , Mingfei Wang, Zhizhen Zhong , Guyue Liu , Ying Zhang, and Kai ChenIn Proceedings of the 19th USENIX Symposium on Operating Systems Design and Implementation (OSDI’25) , 2025
Machine learning (ML) clusters stack multiple network interface cards (NICs) within each server to improve GPU communication bandwidth. However, existing systems fall short in fully utilizing NICs because of statically binding GPU traffic to NICs and PCIe bottleneck. This leads to suboptimal performance under imbalanced traffic, such as when GPUs process different LLM serving requests and training models with varying communication pattern. We propose FuseLink to enable efficient GPU communication over multiple NICs. FuseLink extends inter-server network by integrating high-speed intra-server connections, and recognizes GPUs to efficiently relay traffic to idle NICs. We implement FuseLink and integrate it into NCCL, so that ML applications can use FuseLink seamlessly without code modifications. Compared to NCCL with PXN, we verify that FuseLink can achieve 212GBps bandwidth between two inter-server GPUs and bring speedup on producing the first token in LLM model serving by 1.06-2.89, mixture-of-expert (MoE) training by up to 1.3x, and recommendation model training by up to 1.2x.
- INFOCOMA Generic and Efficient Communication Framework for Message-level In-Network ComputingXinchen Wan , Luyang Li, Han Tian, Xudong Liao , Xinyang Huang, Chaoliang Zeng , Zilong Wang, Xinyu Yang, Ke Cheng, Qingsong Ning , Guyue Liu, Layong Luo, and Kai ChenIn Proceedings of the IEEE International Conference on Computer Communications (INFOCOM’25) , 2025
2024
- EuroSysAccelerating privacy-preserving machine learning with GeniBatchXinyang Huang, Junxue Zhang, Xiaodian Cheng , Hong Zhang, Yilun Jin, Shuihai Hu, Han Tian, and Kai ChenIn Proceedings of the Nineteenth European Conference on Computer Systems (EuroSys’24) , 2024
Cross-silo privacy-preserving machine learning (PPML) adopts Partial Homomorphic Encryption (PHE) for secure data combination and high-quality model training across multiple organizations (e.g., medical and financial). However, PHE introduces significant computation and communication overheads due to data inflation. Batch optimization is an encouraging direction to mitigate the problem by compressing multiple data into a single ciphertext. While promising, it is impractical for a large number of cross-silo PPML applications due to the limited vector operations support and severe data corruption. In this paper, we present GeniBatch, a batch compiler that translates a PPML program with PHE into an efficient program with batch optimization. GeniBatch adopts a set of conversion rules to allow PHE programs involving all vector operations required in cross-silo PPML and ensures end-to-end result consistency before/after compiling. By proposing bit-reserving algorithms, GeniBatch avoids bit-overflow for the correctness of compiled programs and maximizes the compression ratio. We have integrated GeniBatch into FATE, a representative cross-silo PPML framework, and provided SIMD APIs to harness hardware acceleration. Experiments across six popular applications show that GeniBatch achieves up to 22.6x speedup and reduces network traffic by 5.4x-23.8x for generic cross-silo PPML applications.