A Unified Convergence Analysis for Shuffling-Type Gradient Methods

19 February 2020

Papers citing "A Unified Convergence Analysis for Shuffling-Type Gradient Methods"

22 / 22 papers shown

Title
Demystifying SGD with Doubly Stochastic Gradients Kyurae Kim Joohwan Ko Yian Ma Jacob R. Gardner 81 1 0 03 Jun 2024
Distributed Random Reshuffling Methods with Improved Convergence Kun-Yen Huang Linli Zhou Shi Pu 78 4 0 21 Jun 2023
SGD with shuffling: optimal rates without component convexity and large epoch requirements Kwangjun Ahn Chulhee Yun S. Sra 47 66 0 12 Jun 2020
Random Reshuffling: Simple Analysis with Vast Improvements Konstantin Mishchenko Ahmed Khaled Peter Richtárik 62 132 0 10 Jun 2020
Closing the convergence gap of SGD without replacement Shashank Rajput Anant Gupta Dimitris Papailiopoulos 44 61 0 24 Feb 2020
How Good is SGD with Random Shuffling? Itay Safran Ohad Shamir 44 81 0 31 Jul 2019
Momentum-Based Variance Reduction in Non-Convex SGD Ashok Cutkosky Francesco Orabona ODL 70 406 0 24 May 2019
SGD without Replacement: Sharper Rates for General Smooth Convex Functions Prateek Jain Dheeraj M. Nagaraj Praneeth Netrapalli 50 87 0 04 Mar 2019
ProxSARAH: An Efficient Algorithmic Framework for Stochastic Composite Nonconvex Optimization Nhan H. Pham Lam M. Nguyen Dzung Phan Quoc Tran-Dinh 35 140 0 15 Feb 2019
New Convergence Aspects of Stochastic Gradient Algorithms Lam M. Nguyen Phuong Ha Nguyen Peter Richtárik K. Scheinberg Martin Takáč Marten van Dijk 102 66 0 10 Nov 2018
Random Shuffling Beats SGD after Finite Epochs Jeff Z. HaoChen S. Sra 49 98 0 26 Jun 2018
SGD and Hogwild! Convergence Without the Bounded Gradients Assumption Lam M. Nguyen Phuong Ha Nguyen Marten van Dijk Peter Richtárik K. Scheinberg Martin Takáč 66 227 0 11 Feb 2018
Convergence Analysis of Distributed Stochastic Gradient Descent with Shuffling Qi Meng Wei-neng Chen Yue Wang Zhi-Ming Ma Tie-Yan Liu FedML 38 102 0 29 Sep 2017
Variance-Reduced Stochastic Learning under Random Reshuffling Bicheng Ying Kun Yuan Ali H. Sayed 53 13 0 04 Aug 2017
SARAH: A Novel Method for Machine Learning Problems Using Stochastic Recursive Gradient Lam M. Nguyen Jie Liu K. Scheinberg Martin Takáč ODL 157 603 0 01 Mar 2017
Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Łojasiewicz Condition Hamed Karimi J. Nutini Mark Schmidt 242 1,216 0 16 Aug 2016
Optimization Methods for Large-Scale Machine Learning Léon Bottou Frank E. Curtis J. Nocedal 209 3,202 0 15 Jun 2016
Stochastic Variance Reduction for Nonconvex Optimization Sashank J. Reddi Ahmed S. Hefny S. Sra Barnabás Póczós Alex Smola 92 600 0 19 Mar 2016
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 1.3K 149,842 0 22 Dec 2014
Finito: A Faster, Permutable Incremental Gradient Method for Big Data Problems Aaron Defazio T. Caetano Justin Domke 102 169 0 10 Jul 2014
SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives Aaron Defazio Francis R. Bach Simon Lacoste-Julien ODL 128 1,823 0 01 Jul 2014
Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programming Saeed Ghadimi Guanghui Lan ODL 115 1,547 0 22 Sep 2013