Katyusha: The First Direct Acceleration of Stochastic Gradient Methods

18 March 2016

Papers citing "Katyusha: The First Direct Acceleration of Stochastic Gradient Methods"

50 / 297 papers shown

Title
On Structured Filtering-Clustering: Global Error Bound and Optimal First-Order Algorithms Nhat Ho Tianyi Lin Michael I. Jordan 33 2 0 16 Apr 2019
On the Adaptivity of Stochastic Gradient-Based Optimization Lihua Lei Michael I. Jordan ODL 14 22 0 09 Apr 2019
Cocoercivity, Smoothness and Bias in Variance-Reduced Stochastic Gradient Methods Martin Morin Pontus Giselsson 20 2 0 21 Mar 2019
Noisy Accelerated Power Method for Eigenproblems with Applications Vien V. Mai M. Johansson 14 3 0 20 Mar 2019
ProxSARAH: An Efficient Algorithmic Framework for Stochastic Composite Nonconvex Optimization Nhan H. Pham Lam M. Nguyen Dzung Phan Quoc Tran-Dinh 16 139 0 15 Feb 2019
The Complexity of Making the Gradient Small in Stochastic Convex Optimization Dylan J. Foster Ayush Sekhari Ohad Shamir Nathan Srebro Karthik Sridharan Blake E. Woodworth 14 51 0 13 Feb 2019
A Smoother Way to Train Structured Prediction Models Krishna Pillutla Vincent Roulet Sham Kakade Zaïd Harchaoui 19 19 0 08 Feb 2019
Momentum Schemes with Stochastic Variance Reduction for Nonconvex Composite Optimization Yi Zhou Zhe Wang Kaiyi Ji Yingbin Liang Vahid Tarokh ODL 41 14 0 07 Feb 2019
Stochastic first-order methods: non-asymptotic and computer-aided analyses via potential functions Adrien B. Taylor Francis R. Bach 16 60 0 03 Feb 2019
Stochastic Gradient Descent for Nonconvex Learning without Bounded Gradient Assumptions Yunwen Lei Ting Hu Guiying Li K. Tang MLT 29 115 0 03 Feb 2019
Optimal mini-batch and step sizes for SAGA Nidham Gazagnadou Robert Mansel Gower Joseph Salmon 27 34 0 31 Jan 2019
Lower Bounds for Smooth Nonconvex Finite-Sum Optimization Dongruo Zhou Quanquan Gu 21 45 0 31 Jan 2019
Asynchronous Accelerated Proximal Stochastic Gradient for Strongly Convex Distributed Finite Sums Hadrien Hendrikx Francis R. Bach Laurent Massoulié FedML 16 26 0 28 Jan 2019
99% of Distributed Optimization is a Waste of Time: The Issue and How to Fix it Konstantin Mishchenko Filip Hanzely Peter Richtárik 16 13 0 27 Jan 2019
Estimate Sequences for Stochastic Composite Optimization: Variance Reduction, Acceleration, and Robustness to Noise A. Kulunchakov Julien Mairal 32 44 0 25 Jan 2019
Don't Jump Through Hoops and Remove Those Loops: SVRG and Katyusha are Better Without the Outer Loop D. Kovalev Samuel Horváth Peter Richtárik 36 155 0 24 Jan 2019
Curvature-Exploiting Acceleration of Elastic Net Computations Vien V. Mai M. Johansson 22 0 0 24 Jan 2019
Stochastic Trust Region Inexact Newton Method for Large-scale Machine Learning Vinod Kumar Chauhan A. Sharma Kalpana Dahiya 12 6 0 26 Dec 2018
Stochastic Doubly Robust Gradient Kanghoon Lee Jihye Choi Moonsu Cha Jung Kwon Lee Tae-Yoon Kim 13 0 0 21 Dec 2018
On the Ineffectiveness of Variance Reduced Optimization for Deep Learning Aaron Defazio Léon Bottou UQCV DRL 23 112 0 11 Dec 2018
Exploiting Numerical Sparsity for Efficient Learning : Faster Eigenvector Computation and Regression Neha Gupta Aaron Sidford 11 12 0 27 Nov 2018
R-SPIDER: A Fast Riemannian Stochastic Optimization Algorithm with Curvature Independent Rate Jiaming Zhang Hongyi Zhang S. Sra 26 39 0 10 Nov 2018
Accelerating SGD with momentum for over-parameterized learning Chaoyue Liu M. Belkin ODL 4 19 0 31 Oct 2018
Fast and Faster Convergence of SGD for Over-Parameterized Models and an Accelerated Perceptron Sharan Vaswani Francis R. Bach Mark Schmidt 30 296 0 16 Oct 2018
Quasi-hyperbolic momentum and Adam for deep learning Jerry Ma Denis Yarats ODL 84 129 0 16 Oct 2018
ASVRG: Accelerated Proximal SVRG Fanhua Shang L. Jiao Kaiwen Zhou James Cheng Yan Ren Yufei Jin ODL 29 30 0 07 Oct 2018
Continuous-time Models for Stochastic Optimization Algorithms Antonio Orvieto Aurelien Lucchi 19 31 0 05 Oct 2018
Optimal Matrix Momentum Stochastic Approximation and Applications to Q-learning Adithya M. Devraj Ana Bušić Sean P. Meyn 22 4 0 17 Sep 2018
SEGA: Variance Reduction via Gradient Sketching Filip Hanzely Konstantin Mishchenko Peter Richtárik 25 71 0 09 Sep 2018
Online Adaptive Methods, Universality and Acceleration Kfir Y. Levy A. Yurtsever V. Cevher ODL 28 89 0 08 Sep 2018
A Fast Anderson-Chebyshev Acceleration for Nonlinear Optimization Zhize Li Jian Li 18 19 0 07 Sep 2018
Stochastically Controlled Stochastic Gradient for the Convex and Non-convex Composition problem L. Liu Ji Liu Cho-Jui Hsieh Dacheng Tao 14 13 0 06 Sep 2018
Fast Variance Reduction Method with Stochastic Batch Size Xuanqing Liu Cho-Jui Hsieh 20 5 0 07 Aug 2018
Direct Acceleration of SAGA using Sampled Negative Momentum Kaiwen Zhou 13 45 0 28 Jun 2018
A Simple Stochastic Variance Reduced Algorithm with Fast Convergence Rates Kaiwen Zhou Fanhua Shang James Cheng 19 74 0 28 Jun 2018
Stochastic Nested Variance Reduction for Nonconvex Optimization Dongruo Zhou Pan Xu Quanquan Gu 25 146 0 20 Jun 2018
Laplacian Smoothing Gradient Descent Stanley Osher Bao Wang Penghang Yin Xiyang Luo Farzin Barekat Minh Pham A. Lin ODL 22 43 0 17 Jun 2018
Stochastic Gradient Descent with Exponential Convergence Rates of Expected Classification Errors Atsushi Nitanda Taiji Suzuki 24 10 0 14 Jun 2018
Dissipativity Theory for Accelerating Stochastic Variance Reduction: A Unified Analysis of SVRG and Katyusha Using Semidefinite Programs Bin Hu S. Wright Laurent Lessard 27 20 0 10 Jun 2018
Double Quantization for Communication-Efficient Distributed Optimization Yue Yu Jiaxiang Wu Longbo Huang MQ 19 57 0 25 May 2018
Stochastic Gradient Descent for Stochastic Doubly-Nonconvex Composite Optimization Takayuki Kawashima Hironori Fujisawa 14 2 0 21 May 2018
k-SVRG: Variance Reduction for Large Scale Optimization Anant Raj Sebastian U. Stich 10 6 0 02 May 2018
Stochastic Gradient Hamiltonian Monte Carlo with Variance Reduction for Bayesian Inference Zhize Li Tianyi Zhang Shuyu Cheng Jun Yu Li Jian Li BDL 20 18 0 29 Mar 2018
Stochastic model-based minimization of weakly convex functions Damek Davis Dmitriy Drusvyatskiy 35 371 0 17 Mar 2018
On the insufficiency of existing momentum schemes for Stochastic Optimization Rahul Kidambi Praneeth Netrapalli Prateek Jain Sham Kakade ODL 32 117 0 15 Mar 2018
A Stochastic Semismooth Newton Method for Nonsmooth Nonconvex Optimization Andre Milzarek X. Xiao Shicong Cen Zaiwen Wen M. Ulbrich 29 36 0 09 Mar 2018
Not All Samples Are Created Equal: Deep Learning with Importance Sampling Angelos Katharopoulos François Fleuret 23 510 0 02 Mar 2018
VR-SGD: A Simple Stochastic Variance Reduction Method for Machine Learning Fanhua Shang Kaiwen Zhou Hongying Liu James Cheng Ivor W. Tsang Lijun Zhang Dacheng Tao L. Jiao 32 65 0 26 Feb 2018
Guaranteed Sufficient Decrease for Stochastic Variance Reduced Gradient Optimization Fanhua Shang Yuanyuan Liu Kaiwen Zhou James Cheng K. K. Ng Yuichi Yoshida 27 9 0 26 Feb 2018
Differentially Private Empirical Risk Minimization Revisited: Faster and More General Di Wang Minwei Ye Jinhui Xu 19 268 0 14 Feb 2018