LIBRA: Enabling Workload-aware Multi-dimensional Network Topology Optimization for Distributed Training of Large AI Models

24 September 2021

Papers citing "LIBRA: Enabling Workload-aware Multi-dimensional Network Topology Optimization for Distributed Training of Large AI Models"

4 / 4 papers shown

Title
ASTRA-sim2.0: Modeling Hierarchical Networks and Disaggregated Systems for Large-model Training at Scale William Won Taekyung Heo Saeed Rashidi Srinivas Sridharan Sudarshan Srinivasan T. Krishna 36 43 0 24 Mar 2023
Impact of RoCE Congestion Control Policies on Distributed Training of DNNs Tarannum Khan Saeed Rashidi Srinivas Sridharan Pallavi Shurpali Aditya Akella T. Krishna OOD 28 11 0 22 Jul 2022
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 246 4,489 0 23 Jan 2020
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 245 1,821 0 17 Sep 2019