Layered SGD: A Decentralized and Synchronous SGD Algorithm for Scalable
Deep Neural Network Training

Layered SGD: A Decentralized and Synchronous SGD Algorithm for Scalable Deep Neural Network Training

13 June 2019

ArXiv (abs)PDF HTML

Papers citing "Layered SGD: A Decentralized and Synchronous SGD Algorithm for Scalable Deep Neural Network Training"

13 / 13 papers shown

Title
Highly Scalable Deep Learning Training System with Mixed-Precision: Training ImageNet in Four Minutes Xianyan Jia Shutao Song W. He Yangzihao Wang Haidong Rong ... Li Yu Tiegang Chen Guangxiao Hu Shaoshuai Shi Xiaowen Chu 77 384 0 30 Jul 2018
Horovod: fast and easy distributed deep learning in TensorFlow Alexander Sergeev Mike Del Balso 100 1,221 0 15 Feb 2018
Extremely Large Minibatch SGD: Training ResNet-50 on ImageNet in 15 Minutes Takuya Akiba Shuji Suzuki Keisuke Fukuda VLM 48 314 0 12 Nov 2017
Large Batch Training of Convolutional Networks Yang You Igor Gitman Boris Ginsburg ODL 137 852 0 13 Aug 2017
Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour Priya Goyal Piotr Dollár Ross B. Girshick P. Noordhuis Lukasz Wesolowski Aapo Kyrola Andrew Tulloch Yangqing Jia Kaiming He 3DH 128 3,685 0 08 Jun 2017
Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent Xiangru Lian Ce Zhang Huan Zhang Cho-Jui Hsieh Wei Zhang Ji Liu 50 1,233 0 25 May 2017
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.2K 194,322 0 10 Dec 2015
Asynchronous Parallel Stochastic Gradient for Nonconvex Optimization Xiangru Lian Yijun Huang Y. Li Ji Liu 135 499 0 27 Jun 2015
Deep learning with Elastic Averaging SGD Sixin Zhang A. Choromańska Yann LeCun FedML 96 611 0 20 Dec 2014
Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programming Saeed Ghadimi Guanghui Lan ODL 122 1,555 0 22 Sep 2013
Multi-column Deep Neural Networks for Image Classification D. Ciresan U. Meier Jürgen Schmidhuber 168 3,943 0 13 Feb 2012
Making Gradient Descent Optimal for Strongly Convex Stochastic Optimization Alexander Rakhlin Ohad Shamir Karthik Sridharan 169 768 0 26 Sep 2011
Distributed Delayed Stochastic Optimization Alekh Agarwal John C. Duchi 128 627 0 28 Apr 2011