Asynchronous Parallel Stochastic Gradient for Nonconvex Optimization

27 June 2015

Papers citing "Asynchronous Parallel Stochastic Gradient for Nonconvex Optimization"

38 / 88 papers shown

Title
Faster Distributed Deep Net Training: Computation and Communication Decoupled Stochastic Gradient Descent Shuheng Shen Linli Xu Jingchang Liu Xianfeng Liang Yifei Cheng ODL FedML 29 24 0 28 Jun 2019
Fully Decoupled Neural Network Learning Using Delayed Gradients Huiping Zhuang Yi Wang Qinglai Liu Shuai Zhang Zhiping Lin FedML 14 29 0 21 Jun 2019
Layered SGD: A Decentralized and Synchronous SGD Algorithm for Scalable Deep Neural Network Training K. Yu Thomas Flynn Shinjae Yoo N. DÍmperio OffRL 19 6 0 13 Jun 2019
Bandwidth Reduction using Importance Weighted Pruning on Ring AllReduce Zehua Cheng Zhenghua Xu 19 8 0 06 Jan 2019
Asynchronous Stochastic Composition Optimization with Variance Reduction Shuheng Shen Linli Xu Jingchang Liu Junliang Guo Qing Ling 8 2 0 15 Nov 2018
MD-GAN: Multi-Discriminator Generative Adversarial Networks for Distributed Datasets Corentin Hardy Erwan Le Merrer B. Sericola GAN 27 181 0 09 Nov 2018
Toward Understanding the Impact of Staleness in Distributed Machine Learning Wei-Ming Dai Yi Zhou Nanqing Dong Huan Zhang Eric P. Xing 22 80 0 08 Oct 2018
Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithms Jianyu Wang Gauri Joshi 33 348 0 22 Aug 2018
Bayesian Pose Graph Optimization via Bingham Distributions and Tempered Geodesic MCMC Tolga Birdal Umut Simsekli M. Eken Slobodan Ilic 22 38 0 31 May 2018
Double Quantization for Communication-Efficient Distributed Optimization Yue Yu Jiaxiang Wu Longbo Huang MQ 19 57 0 25 May 2018
LAG: Lazily Aggregated Gradient for Communication-Efficient Distributed Learning Tianyi Chen G. Giannakis Tao Sun W. Yin 31 297 0 25 May 2018
Local SGD Converges Fast and Communicates Little Sebastian U. Stich FedML 79 1,044 0 24 May 2018
Taming Convergence for Asynchronous Stochastic Gradient Descent with Unbounded Delay in Non-Convex Learning Xin Zhang Jia-Wei Liu Zhengyuan Zhu 16 17 0 24 May 2018
Tell Me Something New: A New Framework for Asynchronous Parallel Learning Julaiti Alafate Y. Freund FedML 11 2 0 19 May 2018
Parallel and Distributed Successive Convex Approximation Methods for Big-Data Optimization G. Scutari Ying Sun 35 61 0 17 May 2018
Slow and Stale Gradients Can Win the Race: Error-Runtime Trade-offs in Distributed SGD Sanghamitra Dutta Gauri Joshi Soumyadip Ghosh Parijat Dube P. Nagpurkar 31 193 0 03 Mar 2018
Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis Tal Ben-Nun Torsten Hoefler GNN 33 702 0 26 Feb 2018
Asynchronous Stochastic Proximal Methods for Nonconvex Nonsmooth Optimization Rui Zhu Di Niu Zongpeng Li 16 4 0 24 Feb 2018
SparCML: High-Performance Sparse Communication for Machine Learning Cédric Renggli Saleh Ashkboos Mehdi Aghagolzadeh Dan Alistarh Torsten Hoefler 29 126 0 22 Feb 2018
Improved asynchronous parallel optimization analysis for stochastic incremental methods Rémi Leblond Fabian Pedregosa Simon Lacoste-Julien 16 70 0 11 Jan 2018
Efficient Training of Convolutional Neural Nets on Large Distributed Systems Sameer Kumar D. Sreedhar Vaibhav Saxena Yogish Sabharwal Ashish Verma 35 4 0 02 Nov 2017
Gradient Sparsification for Communication-Efficient Distributed Optimization Jianqiao Wangni Jialei Wang Ji Liu Tong Zhang 15 522 0 26 Oct 2017
Convergence Analysis of Distributed Stochastic Gradient Descent with Shuffling Qi Meng Wei-neng Chen Yue Wang Zhi-Ming Ma Tie-Yan Liu FedML 18 101 0 29 Sep 2017
What does fault tolerant Deep Learning need from MPI? Vinay C. Amatya Abhinav Vishnu Charles Siegel J. Daily 28 19 0 11 Sep 2017
On the convergence properties of a $K$ -step averaging stochastic gradient descent algorithm for nonconvex optimization Fan Zhou Guojing Cong 46 232 0 03 Aug 2017
Byzantine-Tolerant Machine Learning Peva Blanchard El-Mahdi El-Mhamdi R. Guerraoui J. Stainer OOD FedML 24 70 0 08 Mar 2017
A Generic Online Parallel Learning Framework for Large Margin Models Shuming Ma Xu Sun FedML 13 2 0 02 Mar 2017
Asynchronous Stochastic Block Coordinate Descent with Variance Reduction Bin Gu Zhouyuan Huo Heng-Chiao Huang 23 10 0 29 Oct 2016
Asynchronous Stochastic Gradient Descent with Delay Compensation Shuxin Zheng Qi Meng Taifeng Wang Wei Chen Nenghai Yu Zhiming Ma Tie-Yan Liu 29 312 0 27 Sep 2016
AdaNet: Adaptive Structural Learning of Artificial Neural Networks Corinna Cortes X. Gonzalvo Vitaly Kuznetsov M. Mohri Scott Yang 29 282 0 05 Jul 2016
Parallel SGD: When does averaging help? Jian Zhang Christopher De Sa Ioannis Mitliagkas Christopher Ré MoMe FedML 54 109 0 23 Jun 2016
ASAGA: Asynchronous Parallel SAGA Rémi Leblond Fabian Pedregosa Simon Lacoste-Julien AI4TS 29 101 0 15 Jun 2016
Omnivore: An Optimizer for Multi-device Deep Learning on CPUs and GPUs Stefan Hadjis Ce Zhang Ioannis Mitliagkas Dan Iter Christopher Ré 20 65 0 14 Jun 2016
Level Up Your Strategy: Towards a Descriptive Framework for Meaningful Enterprise Gamification Xinghao Pan 29 62 0 29 May 2016
DeepSpark: A Spark-Based Distributed Deep Learning Framework for Commodity Clusters Hanjoo Kim Jaehong Park Jaehee Jang Sungroh Yoon BDL 32 37 0 26 Feb 2016
Variance Reduction in SGD by Distributed Importance Sampling Guillaume Alain Alex Lamb Chinnadhurai Sankar Aaron Courville Yoshua Bengio FedML 13 196 0 20 Nov 2015
Model Accuracy and Runtime Tradeoff in Distributed Deep Learning:A Systematic Study Suyog Gupta Wei Zhang Fei Wang 9 170 0 14 Sep 2015
Optimal Distributed Online Prediction using Mini-Batches O. Dekel Ran Gilad-Bachrach Ohad Shamir Lin Xiao 179 683 0 07 Dec 2010