Asynchronous Stochastic Optimization Robust to Arbitrary Delays

22 June 2021

Papers citing "Asynchronous Stochastic Optimization Robust to Arbitrary Delays"

21 / 21 papers shown

Title
Ringmaster ASGD: The First Asynchronous SGD with Optimal Time Complexity Artavazd Maranjyan Alexander Tyurin Peter Richtárik 76 3 0 27 Jan 2025
Is Local SGD Better than Minibatch SGD? Blake E. Woodworth Kumar Kshitij Patel Sebastian U. Stich Zhen Dai Brian Bullins H. B. McMahan Ohad Shamir Nathan Srebro FedML 60 254 0 18 Feb 2020
The Error-Feedback Framework: Better Rates for SGD with Delayed Gradients and Compressed Communication Sebastian U. Stich Sai Praneeth Karimireddy FedML 50 20 0 11 Sep 2019
Error Feedback Fixes SignSGD and other Gradient Compression Schemes Sai Praneeth Karimireddy Quentin Rebjock Sebastian U. Stich Martin Jaggi 54 502 0 28 Jan 2019
A Simple Stochastic Variance Reduced Algorithm with Fast Convergence Rates Kaiwen Zhou Fanhua Shang James Cheng 58 75 0 28 Jun 2018
A Tight Convergence Analysis for Stochastic Gradient Descent with Delayed Updates Yossi Arjevani Ohad Shamir Nathan Srebro 54 63 0 26 Jun 2018
Improved asynchronous parallel optimization analysis for stochastic incremental methods Rémi Leblond Fabian Pedregosa Simon Lacoste-Julien 55 70 0 11 Jan 2018
QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding Dan Alistarh Demjan Grubic Jerry Li Ryota Tomioka Milan Vojnović MQ 64 424 0 07 Oct 2016
Communication-Efficient Learning of Deep Networks from Decentralized Data H. B. McMahan Eider Moore Daniel Ramage S. Hampson Blaise Agüera y Arcas FedML 380 17,437 0 17 Feb 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 1.9K 193,426 0 10 Dec 2015
Asynchronous stochastic convex optimization John C. Duchi Sorathan Chaturapruek Christopher Ré 46 87 0 04 Aug 2015
Perturbed Iterate Analysis for Asynchronous Stochastic Optimization Horia Mania Xinghao Pan Dimitris Papailiopoulos Benjamin Recht Kannan Ramchandran Michael I. Jordan 89 232 0 24 Jul 2015
Asynchronous Parallel Stochastic Gradient for Nonconvex Optimization Xiangru Lian Yijun Huang Y. Li Ji Liu 135 499 0 27 Jun 2015
On Variance Reduction in Stochastic Gradient Descent and its Asynchronous Variants Sashank J. Reddi Ahmed S. Hefny S. Sra Barnabás Póczós Alex Smola 108 196 0 23 Jun 2015
An Asynchronous Mini-Batch Algorithm for Regularized Stochastic Optimization Hamid Reza Feyzmahdavian Arda Aytekin M. Johansson 51 117 0 18 May 2015
Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programming Saeed Ghadimi Guanghui Lan ODL 120 1,547 0 22 Sep 2013
HOGWILD!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent Feng Niu Benjamin Recht Christopher Ré Stephen J. Wright 175 2,273 0 28 Jun 2011
Better Mini-Batch Algorithms via Accelerated Gradient Methods Andrew Cotter Ohad Shamir Nathan Srebro Karthik Sridharan ODL 112 313 0 22 Jun 2011
Distributed Delayed Stochastic Optimization Alekh Agarwal John C. Duchi 123 626 0 28 Apr 2011
Randomized Smoothing for Stochastic Optimization John C. Duchi Peter L. Bartlett Martin J. Wainwright 102 287 0 22 Mar 2011
Optimal Distributed Online Prediction using Mini-Batches O. Dekel Ran Gilad-Bachrach Ohad Shamir Lin Xiao 255 685 0 07 Dec 2010