Hop: Heterogeneity-Aware Decentralized Training

Hop: Heterogeneity-Aware Decentralized Training

4 February 2019

Papers citing "Hop: Heterogeneity-Aware Decentralized Training"

15 / 15 papers shown

Title
Understanding Stragglers in Large Model Training Using What-if Analysis Jinkun Lin Ziheng Jiang Zuquan Song Sida Zhao Menghan Yu ... Shuguang Wang Yanghua Peng Xin Liu Aurojit Panda Jinyang Li 99 1 0 09 May 2025
Communication Compression for Decentralized Training Hanlin Tang Shaoduo Gan Ce Zhang Tong Zhang Ji Liu 63 273 0 17 Mar 2018
Asynchronous Decentralized Parallel Stochastic Gradient Descent Xiangru Lian Wei Zhang Ce Zhang Ji Liu ODL 46 500 0 18 Oct 2017
Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour Priya Goyal Piotr Dollár Ross B. Girshick P. Noordhuis Lukasz Wesolowski Aapo Kyrola Andrew Tulloch Yangqing Jia Kaiming He 3DH 126 3,681 0 08 Jun 2017
Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent Xiangru Lian Ce Zhang Huan Zhang Cho-Jui Hsieh Wei Zhang Ji Liu 50 1,228 0 25 May 2017
ASAP: Asynchronous Approximate Data-Parallel Computation Asim Kadav E. Kruus AI4TS 35 10 0 27 Dec 2016
TensorFlow: A system for large-scale machine learning Martín Abadi P. Barham Jianmin Chen Zhiwen Chen Andy Davis ... Vijay Vasudevan Pete Warden Martin Wicke Yuan Yu Xiaoqiang Zhang GNN AI4CE 433 18,361 0 27 May 2016
Revisiting Distributed Synchronous SGD Jianmin Chen Xinghao Pan R. Monga Samy Bengio Rafal Jozefowicz 87 800 0 04 Apr 2016
MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems Tianqi Chen Mu Li Yutian Li Min Lin Naiyan Wang Minjie Wang Tianjun Xiao Bing Xu Chiyuan Zhang Zheng Zhang 196 2,247 0 03 Dec 2015
High-Performance Distributed ML at Scale through Parameter Server Consistency Models Wei-Ming Dai Abhimanu Kumar Jinliang Wei Qirong Ho Garth A. Gibson Eric Xing 44 120 0 29 Oct 2014
Going Deeper with Convolutions Christian Szegedy Wei Liu Yangqing Jia P. Sermanet Scott E. Reed Dragomir Anguelov D. Erhan Vincent Vanhoucke Andrew Rabinovich 465 43,658 0 17 Sep 2014
Very Deep Convolutional Networks for Large-Scale Image Recognition Karen Simonyan Andrew Zisserman FAtt MDE 1.6K 100,386 0 04 Sep 2014
DimmWitted: A Study of Main-Memory Statistical Analytics Ce Zhang Christopher Ré 122 145 0 28 Mar 2014
HOGWILD!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent Feng Niu Benjamin Recht Christopher Ré Stephen J. Wright 198 2,273 0 28 Jun 2011
Natural Language Processing (almost) from Scratch R. Collobert Jason Weston Léon Bottou Michael Karlen Koray Kavukcuoglu Pavel P. Kuksa 188 7,726 0 02 Mar 2011