Efficient Direct-Connect Topologies for Collective Communications

7 February 2022

Papers citing "Efficient Direct-Connect Topologies for Collective Communications"

5 / 5 papers shown

Title
Efficient All-to-All Collective Communication Schedules for Direct-Connect Topologies P. Basu Liangyu Zhao Jason Fantl Siddharth Pal Arvind Krishnamurthy J. Khoury 35 7 0 24 Sep 2023
TACOS: Topology-Aware Collective Algorithm Synthesizer for Distributed Machine Learning William Won Suvinay Subramanian Sudarshan Srinivasan A. Durg Samvit Kaul Swati Gupta Tushar Krishna 27 6 0 11 Apr 2023
SpectralFly: Ramanujan Graphs as Flexible and Efficient Interconnection Networks Stephen J. Young Sinan G. Aksoy J. Firoz R. Gioiosa Tobias Hagge Mark Kempton Juan Escobedo Mark Raugas 26 5 0 23 Apr 2021
Deep Learning Training in Facebook Data Centers: Design of Scale-up and Scale-out Systems Maxim Naumov John Kim Dheevatsa Mudigere Srinivas Sridharan Xiaodong Wang ... Krishnakumar Nair Isabel Gao Bor-Yiing Su Jiyan Yang M. Smelyanskiy GNN 46 95 0 20 Mar 2020
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 245 1,826 0 17 Sep 2019