Synthesizing Optimal Collective Algorithms

v1v2 (latest)

Synthesizing Optimal Collective Algorithms

19 August 2020

Madan Musuvathi

ArXiv (abs)PDF HTML

Papers citing "Synthesizing Optimal Collective Algorithms"

7 / 7 papers shown

Title
MSCCL++: Rethinking GPU Communication Abstractions for Cutting-edge AI Applications Aashaka Shah Abhinav Jangda Yangqiu Song Caio Rocha Changho Hwang ... Peng Cheng Qinghua Zhou Roshan Dathathri Saeed Maleki Ziyue Yang GNN 96 1 0 11 Apr 2025
Blink: Fast and Generic Collectives for Distributed ML Guanhua Wang Shivaram Venkataraman Amar Phanishayee J. Thelin Nikhil R. Devanur Ion Stoica VLM 54 140 0 11 Oct 2019
Priority-based Parameter Propagation for Distributed DNN Training Anand Jayarajan Jinliang Wei Garth A. Gibson Alexandra Fedorova Gennady Pekhimenko AI4CE 52 179 0 10 May 2019
Evaluating Modern GPU Interconnect: PCIe, NVLink, NV-SLI, NVSwitch and GPUDirect Ang Li Shuaiwen Leon Song Jieyang Chen Jiajia Li Xu Liu Nathan R. Tallent Kevin J. Barker GNN 85 214 0 11 Mar 2019
TicTac: Accelerating Distributed Deep Learning with Communication Scheduling Sayed Hadi Hashemi Sangeetha Abdu Jyothi R. Campbell 41 198 0 08 Mar 2018
Horovod: fast and easy distributed deep learning in TensorFlow Alexander Sergeev Mike Del Balso 100 1,220 0 15 Feb 2018
Poseidon: An Efficient Communication Architecture for Distributed Deep Learning on GPU Clusters Huatian Zhang Zeyu Zheng Shizhen Xu Wei-Ming Dai Qirong Ho Xiaodan Liang Zhiting Hu Jinliang Wei P. Xie Eric Xing GNN 67 345 0 11 Jun 2017

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.