Layered SGD: A Decentralized and Synchronous SGD Algorithm for Scalable
Deep Neural Network Training

Layered SGD: A Decentralized and Synchronous SGD Algorithm for Scalable Deep Neural Network Training

13 June 2019

ArXiv (abs)PDF HTML

Papers citing "Layered SGD: A Decentralized and Synchronous SGD Algorithm for Scalable Deep Neural Network Training"

13 / 13 papers shown

Title
Highly Scalable Deep Learning Training System with Mixed-Precision: Training ImageNet in Four Minutes Xianyan Jia Shutao Song W. He Yangzihao Wang Haidong Rong ... Li Yu Tiegang Chen Guangxiao Hu Shaoshuai Shi Xiaowen Chu 75 384 0 30 Jul 2018
Horovod: fast and easy distributed deep learning in TensorFlow Alexander Sergeev Mike Del Balso 100 1,220 0 15 Feb 2018
Extremely Large Minibatch SGD: Training ResNet-50 on ImageNet in 15 Minutes Takuya Akiba Shuji Suzuki Keisuke Fukuda VLM 48 314 0 12 Nov 2017
Large Batch Training of Convolutional Networks Yang You Igor Gitman Boris Ginsburg ODL 137 848 0 13 Aug 2017
Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour Priya Goyal Piotr Dollár Ross B. Girshick P. Noordhuis Lukasz Wesolowski Aapo Kyrola Andrew Tulloch Yangqing Jia Kaiming He 3DH 128 3,685 0 08 Jun 2017
Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent Xiangru Lian Ce Zhang Huan Zhang Cho-Jui Hsieh Wei Zhang Ji Liu 50 1,228 0 25 May 2017
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.2K 194,322 0 10 Dec 2015
Asynchronous Parallel Stochastic Gradient for Nonconvex Optimization Xiangru Lian Yijun Huang Y. Li Ji Liu 135 499 0 27 Jun 2015
Deep learning with Elastic Averaging SGD Sixin Zhang A. Choromańska Yann LeCun FedML 96 611 0 20 Dec 2014
Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programming Saeed Ghadimi Guanghui Lan ODL 122 1,555 0 22 Sep 2013
Multi-column Deep Neural Networks for Image Classification D. Ciresan U. Meier Jürgen Schmidhuber 168 3,943 0 13 Feb 2012
Making Gradient Descent Optimal for Strongly Convex Stochastic Optimization Alexander Rakhlin Ohad Shamir Karthik Sridharan 169 768 0 26 Sep 2011
Distributed Delayed Stochastic Optimization Alekh Agarwal John C. Duchi 128 627 0 28 Apr 2011

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.