Katyusha: The First Direct Acceleration of Stochastic Gradient Methods

18 March 2016

Papers citing "Katyusha: The First Direct Acceleration of Stochastic Gradient Methods"

50 / 297 papers shown

Title
A Hybrid Stochastic Policy Gradient Algorithm for Reinforcement Learning Nhan H. Pham Lam M. Nguyen Dzung Phan Phuong Ha Nguyen Marten van Dijk Quoc Tran-Dinh 16 25 0 01 Mar 2020
On the Convergence of Nesterov's Accelerated Gradient Method in Stochastic Settings Mahmoud Assran Michael G. Rabbat 19 59 0 27 Feb 2020
On Biased Compression for Distributed Learning Aleksandr Beznosikov Samuel Horváth Peter Richtárik M. Safaryan 10 186 0 27 Feb 2020
Acceleration for Compressed Gradient Descent in Distributed and Federated Optimization Zhize Li D. Kovalev Xun Qian Peter Richtárik FedML AI4CE 29 135 0 26 Feb 2020
Scheduled Restart Momentum for Accelerated Stochastic Gradient Descent Bao Wang T. Nguyen Andrea L. Bertozzi Richard G. Baraniuk Stanley J. Osher ODL 4 48 0 24 Feb 2020
Periodic Q-Learning Dong-hwan Lee Niao He OOD 9 13 0 23 Feb 2020
Adaptivity of Stochastic Gradient Methods for Nonconvex Optimization Samuel Horváth Lihua Lei Peter Richtárik Michael I. Jordan 57 30 0 13 Feb 2020
Variance Reduced Coordinate Descent with Acceleration: New Method With a Surprising Application to Finite-Sum Problems Filip Hanzely D. Kovalev Peter Richtárik 40 17 0 11 Feb 2020
Federated Learning of a Mixture of Global and Local Models Filip Hanzely Peter Richtárik FedML 8 379 0 10 Feb 2020
On the Complexity of Minimizing Convex Finite Sums Without Using the Indices of the Individual Functions Yossi Arjevani Amit Daniely Stefanie Jegelka Hongzhou Lin 24 2 0 09 Feb 2020
Variance Reduction with Sparse Gradients Melih Elibol Lihua Lei Michael I. Jordan 6 23 0 27 Jan 2020
Accelerated Dual-Averaging Primal-Dual Method for Composite Convex Minimization Conghui Tan Yuqiu Qian Shiqian Ma Tong Zhang 20 5 0 15 Jan 2020
Federated Variance-Reduced Stochastic Gradient Descent with Robustness to Byzantine Attacks Zhaoxian Wu Qing Ling Tianyi Chen G. Giannakis FedML AAML 32 181 0 29 Dec 2019
Optimization for deep learning: theory and algorithms Ruoyu Sun ODL 27 168 0 19 Dec 2019
$Support Vector Machine Classifier via $L_{0/1}$ Soft-Margin Loss$ Support Vector Machine Classifier via $L_{0/1}$ Soft-Margin Loss Huajun Wang Yuanhai Shao Shenglong Zhou Ce Zhang N. Xiu VLM 26 51 0 16 Dec 2019
Stochastic Newton and Cubic Newton Methods with Simple Local Linear-Quadratic Rates D. Kovalev Konstantin Mishchenko Peter Richtárik ODL 18 45 0 03 Dec 2019
Efficient Relaxed Gradient Support Pursuit for Sparsity Constrained Non-convex Optimization Fanhua Shang Bingkun Wei Hongying Liu Yuanyuan Liu Jiacheng Zhuo 13 1 0 02 Dec 2019
Exponential Convergence Rates of Classification Errors on Learning with SGD and Random Features Shingo Yashima Atsushi Nitanda Taiji Suzuki 14 2 0 13 Nov 2019
Katyusha Acceleration for Convex Finite-Sum Compositional Optimization Yibo Xu Yangyang Xu 74 13 0 24 Oct 2019
The Practicality of Stochastic Optimization in Imaging Inverse Problems Junqi Tang K. Egiazarian Mohammad Golbabaee Mike Davies 27 30 0 22 Oct 2019
A Stochastic Extra-Step Quasi-Newton Method for Nonsmooth Nonconvex Optimization Minghan Yang Andre Milzarek Zaiwen Wen Tong Zhang ODL 17 36 0 21 Oct 2019
Principal Component Projection and Regression in Nearly Linear Time through Asymmetric SVRG Yujia Jin Aaron Sidford 15 7 0 15 Oct 2019
Randomized Iterative Methods for Linear Systems: Momentum, Inexactness and Gossip Nicolas Loizou 27 5 0 26 Sep 2019
A Stochastic Proximal Point Algorithm for Saddle-Point Problems Luo Luo Cheng Chen Yujun Li Guangzeng Xie Zhihua Zhang 16 16 0 13 Sep 2019
Communication-Efficient Distributed Optimization in Networks with Gradient Tracking and Variance Reduction Boyue Li Shicong Cen Yuxin Chen Yuejie Chi 22 12 0 12 Sep 2019
Decentralized Stochastic Gradient Tracking for Non-convex Empirical Risk Minimization Jiaqi Zhang Keyou You 9 18 0 06 Sep 2019
A General Analysis Framework of Lower Complexity Bounds for Finite-Sum Optimization Guangzeng Xie Luo Luo Zhihua Zhang 14 4 0 22 Aug 2019
Towards Better Generalization: BP-SVRG in Training Deep Neural Networks Hao Jin Dachao Lin Zhihua Zhang ODL 18 2 0 18 Aug 2019
A Data Efficient and Feasible Level Set Method for Stochastic Convex Optimization with Expectation Constraints Qihang Lin Selvaprabu Nadarajah Negar Soheili Tianbao Yang 27 13 0 07 Aug 2019
Towards closing the gap between the theory and practice of SVRG Othmane Sebbouh Nidham Gazagnadou Samy Jelassi Francis R. Bach Robert Mansel Gower 19 17 0 31 Jul 2019
Lookahead Optimizer: k steps forward, 1 step back Michael Ruogu Zhang James Lucas Geoffrey E. Hinton Jimmy Ba ODL 54 723 0 19 Jul 2019
A Hybrid Stochastic Optimization Framework for Stochastic Composite Nonconvex Optimization Quoc Tran-Dinh Nhan H. Pham T. Dzung Lam M. Nguyen 27 49 0 08 Jul 2019
Variance Reduction for Matrix Games Y. Carmon Yujia Jin Aaron Sidford Kevin Tian 13 63 0 03 Jul 2019
Globally Convergent Newton Methods for Ill-conditioned Generalized Self-concordant Losses Ulysse Marteau-Ferey Francis R. Bach Alessandro Rudi 8 35 0 03 Jul 2019
The Role of Memory in Stochastic Optimization Antonio Orvieto Jonas Köhler Aurelien Lucchi 16 30 0 02 Jul 2019
Near-Optimal Methods for Minimizing Star-Convex Functions and Beyond Oliver Hinder Aaron Sidford N. Sohoni 19 70 0 27 Jun 2019
Submodular Batch Selection for Training Deep Neural Networks K. J. Joseph R. VamshiTeja Krishnakant Singh V. Balasubramanian 13 23 0 20 Jun 2019
A Generic Acceleration Framework for Stochastic Composite Optimization A. Kulunchakov Julien Mairal 27 43 0 03 Jun 2019
Unified Acceleration of High-Order Algorithms under Hölder Continuity and Uniform Convexity Chaobing Song Yong Jiang Yi Ma 137 19 0 03 Jun 2019
On the computational complexity of the probabilistic label tree algorithms R. Busa-Fekete Krzysztof Dembczyñski Alexander Golovnev Kalina Jasinska Mikhail Kuznetsov M. Sviridenko Chao Xu TPM 29 3 0 01 Jun 2019
Convergence of Distributed Stochastic Variance Reduced Methods without Sampling Extra Data Shicong Cen Huishuai Zhang Yuejie Chi Wei-neng Chen Tie-Yan Liu FedML 14 27 0 29 May 2019
A unified variance-reduced accelerated gradient method for convex optimization Guanghui Lan Zhize Li Yi Zhou 14 61 0 29 May 2019
Why gradient clipping accelerates training: A theoretical justification for adaptivity Jiaming Zhang Tianxing He S. Sra Ali Jadbabaie 30 446 0 28 May 2019
One Method to Rule Them All: Variance Reduction for Data, Parameters and Many New Methods Filip Hanzely Peter Richtárik 23 26 0 27 May 2019
An Accelerated Decentralized Stochastic Proximal Algorithm for Finite Sums Hadrien Hendrikx Francis R. Bach Laurent Massoulie 24 31 0 27 May 2019
Painless Stochastic Gradient: Interpolation, Line-Search, and Convergence Rates Sharan Vaswani Aaron Mishkin I. Laradji Mark Schmidt Gauthier Gidel Simon Lacoste-Julien ODL 34 205 0 24 May 2019
Hybrid Stochastic Gradient Descent Algorithms for Stochastic Nonconvex Optimization Quoc Tran-Dinh Nhan H. Pham Dzung Phan Lam M. Nguyen 33 54 0 15 May 2019
Solving Empirical Risk Minimization in the Current Matrix Multiplication Time Y. Lee Zhao Song Qiuyi Zhang 19 115 0 11 May 2019
Estimate Sequences for Variance-Reduced Stochastic Composite Optimization A. Kulunchakov Julien Mairal 13 27 0 07 May 2019
Dynamic Mini-batch SGD for Elastic Distributed Training: Learning in the Limbo of Resources Yanghua Peng Hang Zhang Yifei Ma Tong He Zhi-Li Zhang Sheng Zha Mu Li 28 23 0 26 Apr 2019