Katyusha: The First Direct Acceleration of Stochastic Gradient Methods

18 March 2016

Papers citing "Katyusha: The First Direct Acceleration of Stochastic Gradient Methods"

47 / 297 papers shown

Title
A Simple Proximal Stochastic Gradient Method for Nonsmooth Nonconvex Optimization Zhize Li Jian Li 39 116 0 13 Feb 2018
Katyusha X: Practical Momentum Method for Stochastic Sum-of-Nonconvex Optimization Zeyuan Allen-Zhu ODL 46 52 0 12 Feb 2018
Mini-Batch Stochastic ADMMs for Nonconvex Nonsmooth Optimization Feihu Huang Songcan Chen 22 21 0 08 Feb 2018
Linear Convergence of the Primal-Dual Gradient Method for Convex-Concave Saddle Point Problems without Strong Convexity S. Du Wei Hu 68 120 0 05 Feb 2018
How To Make the Gradients Small Stochastically: Even Faster Convex and Nonconvex SGD Zeyuan Allen-Zhu ODL 28 167 0 08 Jan 2018
Momentum and Stochastic Momentum for Stochastic Gradient, Newton, Proximal Point and Subspace Descent Methods Nicolas Loizou Peter Richtárik 24 200 0 27 Dec 2017
The Power of Interpolation: Understanding the Effectiveness of SGD in Modern Over-parametrized Learning Siyuan Ma Raef Bassily M. Belkin 30 287 0 18 Dec 2017
Catalyst Acceleration for First-order Convex Optimization: from Theory to Practice Hongzhou Lin Julien Mairal Zaïd Harchaoui 12 138 0 15 Dec 2017
Asymptotic Analysis via Stochastic Differential Equations of Gradient Descent Algorithms in Statistical and Computational Paradigms Yazhen Wang 14 17 0 27 Nov 2017
Random gradient extrapolation for distributed and stochastic optimization Guanghui Lan Yi Zhou 15 52 0 15 Nov 2017
Duality-free Methods for Stochastic Composition Optimization L. Liu Ji Liu Dacheng Tao 23 16 0 26 Oct 2017
Nesterov's Acceleration For Approximate Newton Haishan Ye Zhihua Zhang ODL 17 13 0 17 Oct 2017
DSCOVR: Randomized Primal-Dual Block Coordinate Algorithms for Asynchronous Distributed Optimization Lin Xiao Adams Wei Yu Qihang Lin Weizhu Chen 14 59 0 13 Oct 2017
First-Order Adaptive Sample Size Methods to Reduce Complexity of Empirical Risk Minimization Aryan Mokhtari Alejandro Ribeiro 14 20 0 02 Sep 2017
Natasha 2: Faster Non-Convex Optimization Than SGD Zeyuan Allen-Zhu ODL 28 245 0 29 Aug 2017
An inexact subsampled proximal Newton-type method for large-scale machine learning Xuanqing Liu Cho-Jui Hsieh Jason D. Lee Yuekai Sun 35 15 0 28 Aug 2017
Accelerated Variance Reduced Stochastic ADMM Yuanyuan Liu Fanhua Shang James Cheng 35 40 0 11 Jul 2017
Stochastic, Distributed and Federated Optimization for Machine Learning Jakub Konecný FedML 29 38 0 04 Jul 2017
Improved Optimization of Finite Sums with Minibatch Stochastic Variance Reduced Proximal Iterations Jialei Wang Tong Zhang 19 12 0 21 Jun 2017
SVM via Saddle Point Optimization: New Bounds and Distributed Algorithms Yifei Jin Lingxiao Huang Jian Li 16 0 0 20 May 2017
Nestrov's Acceleration For Second Order Method Haishan Ye Zhihua Zhang ODL 21 4 0 19 May 2017
Matrix Completion and Related Problems via Strong Duality Maria-Florina Balcan Yingyu Liang David P. Woodruff Hongyang R. Zhang 29 8 0 27 Apr 2017
Accelerating Stochastic Gradient Descent For Least Squares Regression Prateek Jain Sham Kakade Rahul Kidambi Praneeth Netrapalli Aaron Sidford 11 84 0 26 Apr 2017
Larger is Better: The Effect of Learning Rates Enjoyed by Stochastic Optimization with Progressive Variance Reduction Fanhua Shang 17 1 0 17 Apr 2017
Spectrum Approximation Beyond Fast Matrix Multiplication: Algorithms and Hardness Cameron Musco Praneeth Netrapalli Aaron Sidford Shashanka Ubaru David P. Woodruff 8 36 0 13 Apr 2017
Stochastic L-BFGS: Improved Convergence Rates and Practical Acceleration Strategies Renbo Zhao W. Haskell Vincent Y. F. Tan 20 29 0 01 Apr 2017
Catalyst Acceleration for Gradient-Based Non-Convex Optimization Courtney Paquette Hongzhou Lin Dmitriy Drusvyatskiy Julien Mairal Zaïd Harchaoui ODL 29 40 0 31 Mar 2017
Fast Stochastic Variance Reduced Gradient Method with Momentum Acceleration for Machine Learning Fanhua Shang Yuanyuan Liu James Cheng Jiacheng Zhuo ODL 24 23 0 23 Mar 2017
Stochastic Primal Dual Coordinate Method with Non-Uniform Sampling Based on Optimality Violations Atsushi Shibagaki Ichiro Takeuchi 20 5 0 21 Mar 2017
Guaranteed Sufficient Decrease for Variance Reduced Stochastic Gradient Descent Fanhua Shang Yuanyuan Liu James Cheng K. K. Ng Yuichi Yoshida 22 3 0 20 Mar 2017
Doubly Accelerated Stochastic Variance Reduced Dual Averaging Method for Regularized Empirical Risk Minimization Tomoya Murata Taiji Suzuki OffRL 33 28 0 01 Mar 2017
SARAH: A Novel Method for Machine Learning Problems Using Stochastic Recursive Gradient Lam M. Nguyen Jie Liu K. Scheinberg Martin Takáč ODL 42 598 0 01 Mar 2017
Natasha: Faster Non-Convex Stochastic Optimization Via Strongly Non-Convex Parameter Zeyuan Allen-Zhu 23 80 0 02 Feb 2017
Accelerated Variance Reduced Block Coordinate Descent Zebang Shen Hui Qian Chao Zhang Tengfei Zhou 35 1 0 13 Nov 2016
Federated Optimization: Distributed Machine Learning for On-Device Intelligence Jakub Konecný H. B. McMahan Daniel Ramage Peter Richtárik FedML 71 1,877 0 08 Oct 2016
Stochastic Optimization with Variance Reduction for Infinite Datasets with Finite-Sum Structure A. Bietti Julien Mairal 47 36 0 04 Oct 2016
Less than a Single Pass: Stochastically Controlled Stochastic Gradient Method Lihua Lei Michael I. Jordan 29 96 0 12 Sep 2016
Faster Principal Component Regression and Stable Matrix Chebyshev Approximation Zeyuan Allen-Zhu Yuanzhi Li 19 20 0 16 Aug 2016
Doubly Accelerated Methods for Faster CCA and Generalized Eigendecomposition Zeyuan Allen-Zhu Yuanzhi Li 22 50 0 20 Jul 2016
Tight Complexity Bounds for Optimizing Composite Objectives Blake E. Woodworth Nathan Srebro 36 185 0 25 May 2016
Variance Reduction for Faster Non-Convex Optimization Zeyuan Allen-Zhu Elad Hazan ODL 29 390 0 17 Mar 2016
Optimal Black-Box Reductions Between Optimization Objectives Zeyuan Allen-Zhu Elad Hazan 21 96 0 17 Mar 2016
On the Influence of Momentum Acceleration on Online Learning Kun Yuan Bicheng Ying Ali H. Sayed 37 58 0 14 Mar 2016
Stochastic Primal-Dual Coordinate Method for Regularized Empirical Risk Minimization Yuchen Zhang Xiao Lin 43 261 0 10 Sep 2014
A Proximal Stochastic Gradient Method with Progressive Variance Reduction Lin Xiao Tong Zhang ODL 93 737 0 19 Mar 2014
Incremental Majorization-Minimization Optimization with Application to Large-Scale Machine Learning Julien Mairal 79 317 0 18 Feb 2014
A simpler approach to obtaining an O(1/t) convergence rate for the projected stochastic subgradient method Simon Lacoste-Julien Mark Schmidt Francis R. Bach 128 259 0 10 Dec 2012