SGD: General Analysis and Improved Rates

27 January 2019

Papers citing "SGD: General Analysis and Improved Rates"

32 / 82 papers shown

Title
Towards Biologically Plausible Convolutional Networks Roman Pogodin Yash Mehta Timothy Lillicrap P. Latham 26 22 0 22 Jun 2021
Stochastic Polyak Stepsize with a Moving Target Robert Mansel Gower Aaron Defazio Michael G. Rabbat 29 17 0 22 Jun 2021
FedNL: Making Newton-Type Methods Applicable to Federated Learning M. Safaryan Rustem Islamov Xun Qian Peter Richtárik FedML 33 77 0 05 Jun 2021
Spatio-Temporal Neural Network for Fitting and Forecasting COVID-19 Yi-Shuai Niu Wentao Ding Junpeng Hu Wenxu Xu S. Canu 19 2 0 22 Mar 2021
Moshpit SGD: Communication-Efficient Decentralized Training on Heterogeneous Unreliable Devices Max Ryabinin Eduard A. Gorbunov Vsevolod Plokhotnyuk Gennady Pekhimenko 35 32 0 04 Mar 2021
Variance Reduced Training with Stratified Sampling for Forecasting Models Yucheng Lu Youngsuk Park Lifan Chen Bernie Wang Christopher De Sa Dean Phillips Foster AI4TS 38 17 0 02 Mar 2021
IntSGD: Adaptive Floatless Compression of Stochastic Gradients Konstantin Mishchenko Bokun Wang D. Kovalev Peter Richtárik 75 14 0 16 Feb 2021
Local SGD: Unified Theory and New Efficient Methods Eduard A. Gorbunov Filip Hanzely Peter Richtárik FedML 35 109 0 03 Nov 2020
Optimal Client Sampling for Federated Learning Wenlin Chen Samuel Horváth Peter Richtárik FedML 42 191 0 26 Oct 2020
AEGD: Adaptive Gradient Descent with Energy Hailiang Liu Xuping Tian ODL 27 11 0 10 Oct 2020
Optimization for Supervised Machine Learning: Randomized Algorithms for Data and Parameters Filip Hanzely 34 0 0 26 Aug 2020
DBS: Dynamic Batch Size For Distributed Deep Neural Network Training Qing Ye Yuhao Zhou Mingjia Shi Yanan Sun Jiancheng Lv 22 11 0 23 Jul 2020
On stochastic mirror descent with interacting particles: convergence properties and variance reduction Anastasia Borovykh N. Kantas P. Parpas G. Pavliotis 28 12 0 15 Jul 2020
Stochastic Hamiltonian Gradient Methods for Smooth Games Nicolas Loizou Hugo Berard Alexia Jolicoeur-Martineau Pascal Vincent Simon Lacoste-Julien Ioannis Mitliagkas 39 50 0 08 Jul 2020
Federated Learning with Compression: Unified Analysis and Sharp Guarantees Farzin Haddadpour Mohammad Mahdi Kamani Aryan Mokhtari M. Mahdavi FedML 42 273 0 02 Jul 2020
DeltaGrad: Rapid retraining of machine learning models Yinjun Wu Yan Sun S. Davidson MU 25 196 0 26 Jun 2020
SGD for Structured Nonconvex Functions: Learning Rates, Minibatching and Interpolation Robert Mansel Gower Othmane Sebbouh Nicolas Loizou 25 74 0 18 Jun 2020
Random Reshuffling: Simple Analysis with Vast Improvements Konstantin Mishchenko Ahmed Khaled Peter Richtárik 37 131 0 10 Jun 2020
A Unified Theory of Decentralized SGD with Changing Topology and Local Updates Anastasia Koloskova Nicolas Loizou Sadra Boreiri Martin Jaggi Sebastian U. Stich FedML 41 493 0 23 Mar 2020
On the Convergence of Nesterov's Accelerated Gradient Method in Stochastic Settings Mahmoud Assran Michael G. Rabbat 14 59 0 27 Feb 2020
Stochastic Polyak Step-size for SGD: An Adaptive Learning Rate for Fast Convergence Nicolas Loizou Sharan Vaswani I. Laradji Simon Lacoste-Julien 27 181 0 24 Feb 2020
Gradient tracking and variance reduction for decentralized optimization and machine learning Ran Xin S. Kar U. Khan 19 10 0 13 Feb 2020
Adaptivity of Stochastic Gradient Methods for Nonconvex Optimization Samuel Horváth Lihua Lei Peter Richtárik Michael I. Jordan 57 30 0 13 Feb 2020
Variance Reduced Coordinate Descent with Acceleration: New Method With a Surprising Application to Finite-Sum Problems Filip Hanzely D. Kovalev Peter Richtárik 35 17 0 11 Feb 2020
Better Theory for SGD in the Nonconvex World Ahmed Khaled Peter Richtárik 13 178 0 09 Feb 2020
The Practicality of Stochastic Optimization in Imaging Inverse Problems Junqi Tang K. Egiazarian Mohammad Golbabaee Mike Davies 27 30 0 22 Oct 2019
The Error-Feedback Framework: Better Rates for SGD with Delayed Gradients and Compressed Communication Sebastian U. Stich Sai Praneeth Karimireddy FedML 22 20 0 11 Sep 2019
Unified Optimal Analysis of the (Stochastic) Gradient Method Sebastian U. Stich 26 112 0 09 Jul 2019
Beyond Alternating Updates for Matrix Factorization with Inertial Bregman Proximal Gradient Algorithms Mahesh Chandra Mukkamala Peter Ochs 19 21 0 22 May 2019
Characterization of Convex Objective Functions and Optimal Expected Convergence Rates for SGD Marten van Dijk Lam M. Nguyen Phuong Ha Nguyen Dzung Phan 36 6 0 09 Oct 2018
Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Łojasiewicz Condition Hamed Karimi J. Nutini Mark W. Schmidt 139 1,201 0 16 Aug 2016
Stochastic Gradient Descent for Non-smooth Optimization: Convergence Results and Optimal Averaging Schemes Ohad Shamir Tong Zhang 101 571 0 08 Dec 2012