EmbRace: Accelerating Sparse Communication for Distributed Training of
NLP Neural Networks

EmbRace: Accelerating Sparse Communication for Distributed Training of NLP Neural Networks

18 October 2021

Dongsheng Li

Papers citing "EmbRace: Accelerating Sparse Communication for Distributed Training of NLP Neural Networks"

3 / 3 papers shown

Title
Automated Tensor Model Parallelism with Overlapped Communication for Efficient Foundation Model Training Shengwei Li Zhiquan Lai Yanqi Hao Weijie Liu Ke-shi Ge Xiaoge Deng Dongsheng Li KaiCheng Lu 21 10 0 25 May 2023
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu M. Schuster Zhehuai Chen Quoc V. Le Mohammad Norouzi ... Alex Rudnick Oriol Vinyals G. Corrado Macduff Hughes J. Dean AIMat 718 6,750 0 26 Sep 2016
Distributed Training of Deep Neural Networks: Theoretical and Practical Limits of Parallel Scalability J. Keuper Franz-Josef Pfreundt GNN 55 97 0 22 Sep 2016