Momentum and Stochastic Momentum for Stochastic Gradient, Newton, Proximal Point and Subspace Descent Methods

27 December 2017

Papers citing "Momentum and Stochastic Momentum for Stochastic Gradient, Newton, Proximal Point and Subspace Descent Methods"

44 / 44 papers shown

Title
HOME-3: High-Order Momentum Estimator with Third-Power Gradient for Convex and Smooth Nonconvex Optimization Wei Zhang Arif Hassan Zidan Afrar Jahin Wei Zhang Tianming Liu ODL 55 0 0 16 May 2025
Stochastic Polyak Step-sizes and Momentum: Convergence Guarantees and Practical Performance Dimitris Oikonomou Nicolas Loizou 66 6 0 06 Jun 2024
SGD for Structured Nonconvex Functions: Learning Rates, Minibatching and Interpolation Robert Mansel Gower Othmane Sebbouh Nicolas Loizou 81 75 0 18 Jun 2020
Reducing the variance in online optimization by transporting past gradients Sébastien M. R. Arnold Pierre-Antoine Manzagol Reza Babanezhad Ioannis Mitliagkas Nicolas Le Roux 61 28 0 08 Jun 2019
Revisiting Randomized Gossip Algorithms: General Framework, Convergence Rates and Novel Block and Accelerated Protocols Nicolas Loizou Peter Richtárik 45 35 0 20 May 2019
Convergence Analysis of Inexact Randomized Iterative Methods Nicolas Loizou Peter Richtárik 50 21 0 19 Mar 2019
A Privacy Preserving Randomized Gossip Algorithm via Controlled Noise Insertion Filip Hanzely Jakub Konecný Nicolas Loizou Peter Richtárik Dmitry Grishchenko 39 10 0 27 Jan 2019
Don't Jump Through Hoops and Remove Those Loops: SVRG and Katyusha are Better Without the Outer Loop D. Kovalev Samuel Horváth Peter Richtárik 86 156 0 24 Jan 2019
Accelerated Linear Convergence of Stochastic Momentum Methods in Wasserstein Distances Bugra Can Mert Gurbuzbalaban Lingjiong Zhu 48 45 0 22 Jan 2019
Provably Accelerated Randomized Gossip Algorithms Nicolas Loizou Michael G. Rabbat Peter Richtárik 42 19 0 31 Oct 2018
Quasi-hyperbolic momentum and Adam for deep learning Jerry Ma Denis Yarats ODL 123 130 0 16 Oct 2018
Accelerated Gossip via Stochastic Heavy Ball Method Nicolas Loizou Peter Richtárik 38 27 0 23 Sep 2018
Optimal Matrix Momentum Stochastic Approximation and Applications to Q-learning Adithya M. Devraj Ana Bušić Sean P. Meyn 114 4 0 17 Sep 2018
Direct Acceleration of SAGA using Sampled Negative Momentum Kaiwen Zhou 53 45 0 28 Jun 2018
A Simple Stochastic Variance Reduced Algorithm with Fast Convergence Rates Kaiwen Zhou Fanhua Shang James Cheng 60 75 0 28 Jun 2018
Linearly convergent stochastic heavy ball method for minimizing generalization error Nicolas Loizou Peter Richtárik 106 45 0 30 Oct 2017
Accelerated Stochastic Power Iteration Christopher De Sa Bryan D. He Ioannis Mitliagkas Christopher Ré Peng Xu 72 90 0 10 Jul 2017
Stochastic Primal-Dual Hybrid Gradient Algorithm with Arbitrary Sampling and Imaging Applications A. Chambolle Matthias Joachim Ehrhardt Peter Richtárik Carola-Bibiane Schönlieb 80 185 0 15 Jun 2017
YellowFin and the Art of Momentum Tuning Jian Zhang Ioannis Mitliagkas ODL 49 108 0 12 Jun 2017
Stochastic Reformulations of Linear Systems: Algorithms and Convergence Theory Peter Richtárik Martin Takáč 61 93 0 04 Jun 2017
The Marginal Value of Adaptive Gradient Methods in Machine Learning Ashia Wilson Rebecca Roelofs Mitchell Stern Nathan Srebro Benjamin Recht ODL 58 1,029 0 23 May 2017
A New Perspective on Randomized Gossip Algorithms Nicolas Loizou Peter Richtárik 36 30 0 15 Oct 2016
Stochastic Heavy Ball S. Gadat Fabien Panloup Sofiane Saadane 93 103 0 14 Sep 2016
Unified Convergence Analysis of Stochastic Momentum Methods for Convex and Non-convex Optimization Tianbao Yang Qihang Lin Zhe Li 59 122 0 12 Apr 2016
Katyusha: The First Direct Acceleration of Stochastic Gradient Methods Zeyuan Allen-Zhu ODL 96 580 0 18 Mar 2016
A Simple Practical Accelerated Method for Finite Sums Aaron Defazio 117 121 0 08 Feb 2016
Even Faster Accelerated Coordinate Descent Using Non-Uniform Sampling Zeyuan Allen-Zhu Zheng Qu Peter Richtárik Yang Yuan 74 171 0 30 Dec 2015
Coordinate Descent Converges Faster with the Gauss-Southwell Rule Than Random Selection J. Nutini Mark Schmidt I. Laradji M. Friedlander H. Koepke 66 223 0 01 Jun 2015
Mini-Batch Semi-Stochastic Gradient Descent in the Proximal Setting Jakub Konecný Jie Liu Peter Richtárik Martin Takáč ODL 97 273 0 16 Apr 2015
SDNA: Stochastic Dual Newton Ascent for Empirical Risk Minimization Zheng Qu Peter Richtárik Martin Takáč Olivier Fercoq ODL 76 99 0 08 Feb 2015
Coordinate Descent with Arbitrary Sampling II: Expected Separable Overapproximation Zheng Qu Peter Richtárik 156 83 0 27 Dec 2014
Coordinate Descent with Arbitrary Sampling I: Algorithms and Complexity Zheng Qu Peter Richtárik 69 130 0 27 Dec 2014
Going Deeper with Convolutions Christian Szegedy Wei Liu Yangqing Jia P. Sermanet Scott E. Reed Dragomir Anguelov D. Erhan Vincent Vanhoucke Andrew Rabinovich 457 43,649 0 17 Sep 2014
SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives Aaron Defazio Francis R. Bach Simon Lacoste-Julien ODL 131 1,823 0 01 Jul 2014
iPiano: Inertial Proximal Algorithm for Non-Convex Optimization Peter Ochs Yunjin Chen Thomas Brox Thomas Pock 73 432 0 18 Apr 2014
Accelerated, Parallel and Proximal Coordinate Descent Olivier Fercoq Peter Richtárik 88 372 0 20 Dec 2013
Semi-Stochastic Gradient Descent Methods Jakub Konecný Peter Richtárik ODL 113 238 0 05 Dec 2013
Stochastic Gradient Descent, Weighted Sampling, and the Randomized Kaczmarz algorithm Deanna Needell Nathan Srebro Rachel A. Ward 134 553 0 21 Oct 2013
Minimizing Finite Sums with the Stochastic Average Gradient Mark Schmidt Nicolas Le Roux Francis R. Bach 316 1,246 0 10 Sep 2013
Parallel Coordinate Descent Methods for Big Data Optimization Peter Richtárik Martin Takáč 125 487 0 04 Dec 2012
Accelerated Gradient Methods for Networked Optimization E. Ghadimi Iman Shames M. Johansson 175 96 0 09 Nov 2012
Stochastic Dual Coordinate Ascent Methods for Regularized Loss Minimization Shai Shalev-Shwartz Tong Zhang 178 1,033 0 10 Sep 2012
Iteration Complexity of Randomized Block-Coordinate Descent Methods for Minimizing a Composite Function Peter Richtárik Martin Takáč 91 770 0 14 Jul 2011
Gossip Algorithms for Distributed Signal Processing A. Dimakis S. Kar José M. F. Moura Michael G. Rabbat Anna Scaglione 132 857 0 27 Mar 2010