Katyusha: The First Direct Acceleration of Stochastic Gradient Methods

18 March 2016

Papers citing "Katyusha: The First Direct Acceleration of Stochastic Gradient Methods"

50 / 191 papers shown

Title
HOME-3: High-Order Momentum Estimator with Third-Power Gradient for Convex and Smooth Nonconvex Optimization Wei Zhang Arif Hassan Zidan Afrar Jahin Wei Zhang Tianming Liu ODL 101 0 0 16 May 2025
Tensor Sketch: Fast and Scalable Polynomial Kernel Approximation Ninh Pham Rasmus Pagh 140 0 0 13 May 2025
Trial and Trust: Addressing Byzantine Attacks with Comprehensive Defense Strategy Gleb Molodtsov Daniil Medyakov Sergey Skorik Nikolas Khachaturov Shahane Tigranyan Vladimir Aletov A. Avetisyan Martin Takáč Aleksandr Beznosikov AAML 126 0 0 12 May 2025
Accelerated Distributed Optimization with Compression and Error Feedback Yuan Gao Anton Rodomanov Jeremy Rack Sebastian U. Stich 91 0 0 11 Mar 2025
Variance Reduction Methods Do Not Need to Compute Full Gradients: Improved Efficiency through Shuffling Daniil Medyakov Gleb Molodtsov S. Chezhegov Alexey Rebrikov Aleksandr Beznosikov 151 0 0 21 Feb 2025
SAPPHIRE: Preconditioned Stochastic Variance Reduction for Faster Large-Scale Statistical Learning Jingruo Sun Zachary Frangella Madeleine Udell 84 0 0 28 Jan 2025
Gradient-Free Method for Heavily Constrained Nonconvex Optimization Wanli Shi Hongchang Gao Bin Gu 116 5 0 31 Aug 2024
Fine-grained Analysis and Faster Algorithms for Iteratively Solving Linear Systems Michal Dereziñski Daniel LeJeune Deanna Needell E. Rebrova 88 4 0 09 May 2024
Fast Stochastic Policy Gradient: Negative Momentum for Reinforcement Learning Haobin Zhang Zhuang Yang 70 0 0 08 May 2024
Second-order Information Promotes Mini-Batch Robustness in Variance-Reduced Gradients Sachin Garg A. Berahas Michal Dereziñski 100 1 0 23 Apr 2024
A Coefficient Makes SVRG Effective Yida Yin Zhiqiu Xu Zhiyuan Li Trevor Darrell Zhuang Liu 105 1 0 09 Nov 2023
Random feature approximation for general spectral methods Mike Nguyen Nicole Mücke 62 1 0 29 Aug 2023
Nonconvex Stochastic Bregman Proximal Gradient Method with Application to Deep Learning Kuan-Fu Ding Jingyang Li Kim-Chuan Toh 142 8 0 26 Jun 2023
Sarah Frank-Wolfe: Methods for Constrained Optimization with Best Rates and Practical Features Aleksandr Beznosikov David Dobre Gauthier Gidel 97 6 0 23 Apr 2023
Stochastic Distributed Optimization under Average Second-order Similarity: Algorithms and Analysis Dachao Lin Yuze Han Haishan Ye Zhihua Zhang 99 12 0 15 Apr 2023
Sharper Analysis for Minibatch Stochastic Proximal Point Methods: Stability, Smoothness, and Deviation Xiao-Tong Yuan P. Li 93 2 0 09 Jan 2023
Variance-Reduced Conservative Policy Iteration Naman Agarwal Brian Bullins Karan Singh 64 3 0 12 Dec 2022
Cyclic Block Coordinate Descent With Variance Reduction for Composite Nonconvex Optimization Xu Cai Chaobing Song Stephen J. Wright Jelena Diakonikolas 87 14 0 09 Dec 2022
Statistical and Computational Guarantees for Influence Diagnostics Jillian R. Fisher Lang Liu Krishna Pillutla Y. Choi Zaïd Harchaoui TDI 74 0 0 08 Dec 2022
Stochastic Steffensen method Minda Zhao Zehua Lai Lek-Heng Lim ODL 56 4 0 28 Nov 2022
Accelerated Riemannian Optimization: Handling Constraints with a Prox to Bound Geometric Penalties David Martínez-Rubio Sebastian Pokutta 93 10 0 26 Nov 2022
Flatter, faster: scaling momentum for optimal speedup of SGD Aditya Cowsik T. Can Paolo Glorioso 122 5 0 28 Oct 2022
Faster federated optimization under second-order similarity Ahmed Khaled Chi Jin FedML 100 19 0 06 Sep 2022
Simple and Optimal Stochastic Gradient Methods for Nonsmooth Nonconvex Optimization Zhize Li Jian Li 115 6 0 22 Aug 2022
Adaptive Learning Rates for Faster Stochastic Gradient Methods Samuel Horváth Konstantin Mishchenko Peter Richtárik ODL 65 9 0 10 Aug 2022
SGEM: stochastic gradient with energy and momentum Hailiang Liu Xuping Tian 51 4 0 03 Aug 2022
RECAPP: Crafting a More Efficient Catalyst for Convex Optimization Y. Carmon A. Jambulapati Yujia Jin Aaron Sidford 129 12 0 17 Jun 2022
On the fast convergence of minibatch heavy ball momentum Raghu Bollapragada Tyler Chen Rachel A. Ward 122 19 0 15 Jun 2022
Distributed Newton-Type Methods with Communication Compression and Bernoulli Aggregation Rustem Islamov Xun Qian Slavomír Hanzely M. Safaryan Peter Richtárik 79 16 0 07 Jun 2022
Stochastic Variance-Reduced Newton: Accelerating Finite-Sum Minimization with Large Batches Michal Derezinski 145 6 0 06 Jun 2022
Trajectory of Mini-Batch Momentum: Batch Size Saturation and Convergence in High Dimensions Kiwon Lee Andrew N. Cheng Courtney Paquette Elliot Paquette 87 14 0 02 Jun 2022
Bring Your Own Algorithm for Optimal Differentially Private Stochastic Minimax Optimization Liang Zhang K. K. Thekumparampil Sewoong Oh Niao He 96 20 0 01 Jun 2022
How catastrophic can catastrophic forgetting be in linear regression? Itay Evron E. Moroshko Rachel A. Ward Nati Srebro Daniel Soudry CLL 88 54 0 19 May 2022
An Adaptive Incremental Gradient Method With Support for Non-Euclidean Norms Binghui Xie Chen Jin Kaiwen Zhou James Cheng Wei Meng 76 1 0 28 Apr 2022
Distributionally Robust Optimization via Ball Oracle Acceleration Y. Carmon Danielle Hausler 75 13 0 24 Mar 2022
Stochastic Halpern Iteration with Variance Reduction for Stochastic Monotone Inclusions Xu Cai Chaobing Song Cristóbal Guzmán Jelena Diakonikolas 114 11 0 17 Mar 2022
An Accelerated Stochastic Algorithm for Solving the Optimal Transport Problem Yiling Xie Yiling Luo X. Huo 104 11 0 02 Mar 2022
Data-Consistent Local Superresolution for Medical Imaging Junqi Tang SupR 77 0 0 22 Feb 2022
Sharper Rates for Separable Minimax and Finite Sum Optimization via Primal-Dual Extragradient Methods Yujia Jin Aaron Sidford Kevin Tian 87 31 0 09 Feb 2022
Decentralized Stochastic Variance Reduced Extragradient Method Luo Luo Haishan Ye 114 7 0 01 Feb 2022
PAGE-PG: A Simple and Loopless Variance-Reduced Policy Gradient Method with Probabilistic Gradient Estimation Matilde Gargiani Andrea Zanelli Andrea Martinelli Tyler H. Summers John Lygeros 78 14 0 01 Feb 2022
L-SVRG and L-Katyusha with Adaptive Sampling Boxin Zhao Boxiang Lyu Mladen Kolar 86 3 0 31 Jan 2022
On the Complexity of a Practical Primal-Dual Coordinate Method Ahmet Alacaoglu Volkan Cevher Stephen J. Wright 83 13 0 19 Jan 2022
Adaptive Client Sampling in Federated Learning via Online Learning with Bandit Feedback Boxin Zhao Lingxiao Wang Mladen Kolar Ziqi Liu Qing Cui Jun Zhou Chaochao Chen FedML 176 11 0 28 Dec 2021
Random-reshuffled SARAH does not need a full gradient computations Aleksandr Beznosikov Martin Takáč 81 8 0 26 Nov 2021
EF21 with Bells & Whistles: Six Algorithmic Extensions of Modern Error Feedback Ilyas Fatkhullin Igor Sokolov Eduard A. Gorbunov Zhize Li Peter Richtárik 134 47 0 07 Oct 2021
Accelerating Perturbed Stochastic Iterates in Asynchronous Lock-Free Optimization Kaiwen Zhou Anthony Man-Cho So James Cheng 76 1 0 30 Sep 2021
Asynchronous Iterations in Optimization: New Sequence Results and Sharper Algorithmic Guarantees Hamid Reza Feyzmahdavian M. Johansson 85 21 0 09 Sep 2021
Robust Regression Revisited: Acceleration and Improved Estimation Rates A. Jambulapati Jingkai Li T. Schramm Kevin Tian AAML 82 17 0 22 Jun 2021
Kernel Clustering with Sigmoid-based Regularization for Efficient Segmentation of Sequential Data Tung Doan Atsuhiro Takasu 91 1 0 22 Jun 2021