Katyusha: The First Direct Acceleration of Stochastic Gradient Methods

18 March 2016

Papers citing "Katyusha: The First Direct Acceleration of Stochastic Gradient Methods"

50 / 297 papers shown

Title
Flatter, faster: scaling momentum for optimal speedup of SGD Aditya Cowsik T. Can Paolo Glorioso 62 5 0 28 Oct 2022
Faster federated optimization under second-order similarity Ahmed Khaled Chi Jin FedML 56 18 0 06 Sep 2022
Simple and Optimal Stochastic Gradient Methods for Nonsmooth Nonconvex Optimization Zhize Li Jian Li 43 6 0 22 Aug 2022
Adaptive Learning Rates for Faster Stochastic Gradient Methods Samuel Horváth Konstantin Mishchenko Peter Richtárik ODL 41 7 0 10 Aug 2022
Decomposable Non-Smooth Convex Optimization with Nearly-Linear Gradient Oracle Complexity Sally Dong Haotian Jiang Y. Lee Swati Padmanabhan Guanghao Ye 33 2 0 07 Aug 2022
SGEM: stochastic gradient with energy and momentum Hailiang Liu Xuping Tian 22 4 0 03 Aug 2022
Stochastic Primal-Dual Three Operator Splitting Algorithm with Extension to Equivariant Regularization-by-Denoising Junqi Tang Matthias Joachim Ehrhardt Carola-Bibiane Schönlieb 36 0 0 02 Aug 2022
RECAPP: Crafting a More Efficient Catalyst for Convex Optimization Y. Carmon A. Jambulapati Yujia Jin Aaron Sidford 60 11 0 17 Jun 2022
On the fast convergence of minibatch heavy ball momentum Raghu Bollapragada Tyler Chen Rachel A. Ward 39 17 0 15 Jun 2022
A Unified Approach to Reinforcement Learning, Quantal Response Equilibria, and Two-Player Zero-Sum Games Samuel Sokota Ryan DÓrazio J. Zico Kolter Nicolas Loizou Marc Lanctot Ioannis Mitliagkas Noam Brown Christian Kroer 36 1 0 12 Jun 2022
Distributed Newton-Type Methods with Communication Compression and Bernoulli Aggregation Rustem Islamov Xun Qian Slavomír Hanzely M. Safaryan Peter Richtárik 40 16 0 07 Jun 2022
Stochastic Variance-Reduced Newton: Accelerating Finite-Sum Minimization with Large Batches Michal Derezinski 55 6 0 06 Jun 2022
Trajectory of Mini-Batch Momentum: Batch Size Saturation and Convergence in High Dimensions Kiwon Lee Andrew N. Cheng Courtney Paquette Elliot Paquette 24 12 0 02 Jun 2022
Variance Reduction is an Antidote to Byzantines: Better Rates, Weaker Assumptions and Communication Compression as a Cherry on the Top Eduard A. Gorbunov Samuel Horváth Peter Richtárik Gauthier Gidel AAML 19 0 0 01 Jun 2022
Bring Your Own Algorithm for Optimal Differentially Private Stochastic Minimax Optimization Liang Zhang K. K. Thekumparampil Sewoong Oh Niao He 49 17 0 01 Jun 2022
SADAM: Stochastic Adam, A Stochastic Operator for First-Order Gradient-based Optimizer Wei Zhang Yun-Jian Bao ODL 33 2 0 20 May 2022
How catastrophic can catastrophic forgetting be in linear regression? Itay Evron E. Moroshko Rachel A. Ward Nati Srebro Daniel Soudry CLL 35 48 0 19 May 2022
Implicit Regularization Properties of Variance Reduced Stochastic Mirror Descent Yiling Luo X. Huo Y. Mei 23 1 0 29 Apr 2022
An Adaptive Incremental Gradient Method With Support for Non-Euclidean Norms Binghui Xie Chen Jin Kaiwen Zhou James Cheng Wei Meng 40 1 0 28 Apr 2022
Distributionally Robust Optimization via Ball Oracle Acceleration Y. Carmon Danielle Hausler 20 12 0 24 Mar 2022
An Adaptive Gradient Method with Energy and Momentum Hailiang Liu Xuping Tian ODL 21 9 0 23 Mar 2022
Provable Constrained Stochastic Convex Optimization with XOR-Projected Gradient Descent Fan Ding Yijie Wang Jianzhu Ma Yexiang Xue 25 0 0 22 Mar 2022
Stochastic Halpern Iteration with Variance Reduction for Stochastic Monotone Inclusions Xu Cai Chaobing Song Cristóbal Guzmán Jelena Diakonikolas 48 9 0 17 Mar 2022
Accelerated SGD for Non-Strongly-Convex Least Squares Aditya Varre Nicolas Flammarion 26 7 0 03 Mar 2022
An Accelerated Stochastic Algorithm for Solving the Optimal Transport Problem Yiling Xie Yiling Luo X. Huo 29 10 0 02 Mar 2022
Data-Consistent Local Superresolution for Medical Imaging Junqi Tang SupR 33 0 0 22 Feb 2022
Equivariance Regularization for Image Reconstruction Junqi Tang 32 2 0 10 Feb 2022
Sharper Rates for Separable Minimax and Finite Sum Optimization via Primal-Dual Extragradient Methods Yujia Jin Aaron Sidford Kevin Tian 21 30 0 09 Feb 2022
Decentralized Stochastic Variance Reduced Extragradient Method Luo Luo Haishan Ye 27 7 0 01 Feb 2022
PAGE-PG: A Simple and Loopless Variance-Reduced Policy Gradient Method with Probabilistic Gradient Estimation Matilde Gargiani Andrea Zanelli Andrea Martinelli Tyler H. Summers John Lygeros 33 14 0 01 Feb 2022
L-SVRG and L-Katyusha with Adaptive Sampling Boxin Zhao Boxiang Lyu Mladen Kolar 26 3 0 31 Jan 2022
Adaptive Accelerated (Extra-)Gradient Methods with Variance Reduction Zijian Liu Ta Duy Nguyen Alina Ene Huy Le Nguyen 30 6 0 28 Jan 2022
On the Complexity of a Practical Primal-Dual Coordinate Method Ahmet Alacaoglu V. Cevher Stephen J. Wright 23 12 0 19 Jan 2022
Adaptive Client Sampling in Federated Learning via Online Learning with Bandit Feedback Boxin Zhao Lingxiao Wang Mladen Kolar Ziqi Liu Qing Cui Jun Zhou Chaochao Chen FedML 39 10 0 28 Dec 2021
Random-reshuffled SARAH does not need a full gradient computations Aleksandr Beznosikov Martin Takáč 31 7 0 26 Nov 2021
Coordinate Linear Variance Reduction for Generalized Linear Programming Chaobing Song Cheuk Yin Lin Stephen J. Wright Jelena Diakonikolas 32 12 0 02 Nov 2021
Stochastic Primal-Dual Deep Unrolling Junqi Tang Subhadip Mukherjee Carola-Bibiane Schönlieb 24 4 0 19 Oct 2021
Training Deep Neural Networks with Adaptive Momentum Inspired by the Quadratic Optimization Tao Sun Huaming Ling Zuoqiang Shi Dongsheng Li Bao Wang ODL 27 13 0 18 Oct 2021
Distributed Methods with Compressed Communication for Solving Variational Inequalities, with Theoretical Guarantees Aleksandr Beznosikov Peter Richtárik Michael Diskin Max Ryabinin Alexander Gasnikov FedML 25 21 0 07 Oct 2021
EF21 with Bells & Whistles: Practical Algorithmic Extensions of Modern Error Feedback Ilyas Fatkhullin Igor Sokolov Eduard A. Gorbunov Zhize Li Peter Richtárik 51 46 0 07 Oct 2021
Accelerating Perturbed Stochastic Iterates in Asynchronous Lock-Free Optimization Kaiwen Zhou Anthony Man-Cho So James Cheng 27 1 0 30 Sep 2021
Asynchronous Iterations in Optimization: New Sequence Results and Sharper Algorithmic Guarantees Hamid Reza Feyzmahdavian M. Johansson 32 20 0 09 Sep 2021
FedPAGE: A Fast Local Stochastic Gradient Method for Communication-Efficient Federated Learning Haoyu Zhao Zhize Li Peter Richtárik FedML 30 29 0 10 Aug 2021
Distributed stochastic inertial-accelerated methods with delayed derivatives for nonconvex problems Yangyang Xu Yibo Xu Yonggui Yan Jiewei Chen 27 4 0 24 Jul 2021
CANITA: Faster Rates for Distributed Convex Optimization with Communication Compression Zhize Li Peter Richtárik 33 29 0 20 Jul 2021
Robust Regression Revisited: Acceleration and Improved Estimation Rates A. Jambulapati Jingkai Li T. Schramm Kevin Tian AAML 32 17 0 22 Jun 2021
Kernel Clustering with Sigmoid-based Regularization for Efficient Segmentation of Sequential Data Tung Doan Atsuhiro Takasu 43 1 0 22 Jun 2021
Stochastic Bias-Reduced Gradient Methods Hilal Asi Y. Carmon A. Jambulapati Yujia Jin Aaron Sidford 31 29 0 17 Jun 2021
Exploiting Local Convergence of Quasi-Newton Methods Globally: Adaptive Sample Size Approach Qiujiang Jin Aryan Mokhtari 16 4 0 10 Jun 2021
Dynamics of Stochastic Momentum Methods on Large-scale, Quadratic Models Courtney Paquette Elliot Paquette ODL 32 13 0 07 Jun 2021