On the fast convergence of minibatch heavy ball momentum

15 June 2022

Papers citing "On the fast convergence of minibatch heavy ball momentum"

10 / 10 papers shown

Title
Analysis of an Idealized Stochastic Polyak Method and its Application to Black-Box Model Distillation Robert M. Gower Guillaume Garrigos Nicolas Loizou Dimitris Oikonomou Konstantin Mishchenko Fabian Schaipp 33 0 0 02 Apr 2025
Fine-grained Analysis and Faster Algorithms for Iteratively Solving Linear Systems Michal Dereziñski Daniel LeJeune Deanna Needell E. Rebrova 42 3 0 09 May 2024
Randomized Kaczmarz with geometrically smoothed momentum Seth J. Alderman Roan W. Luikart Nicholas F. Marshall 24 2 0 17 Jan 2024
(Accelerated) Noise-adaptive Stochastic Heavy-Ball Momentum Anh Dang Reza Babanezhad Sharan Vaswani 27 0 0 12 Jan 2024
Accelerated Convergence of Stochastic Heavy Ball Method under Anisotropic Gradient Noise Rui Pan Yuxing Liu Xiaoyu Wang Tong Zhang 26 5 0 22 Dec 2023
When and Why Momentum Accelerates SGD:An Empirical Study Jingwen Fu Bohan Wang Huishuai Zhang Zhizheng Zhang Wei Chen Na Zheng 22 10 0 15 Jun 2023
Fast Diffusion Model Zike Wu Pan Zhou Kenji Kawaguchi Hanwang Zhang DiffM 21 19 0 12 Jun 2023
Acceleration of stochastic gradient descent with momentum by averaging: finite-sample rates and asymptotic normality Kejie Tang Weidong Liu Yichen Zhang Xi Chen 21 2 0 28 May 2023
Trajectory of Mini-Batch Momentum: Batch Size Saturation and Convergence in High Dimensions Kiwon Lee Andrew N. Cheng Courtney Paquette Elliot Paquette 11 12 0 02 Jun 2022
A Simple Convergence Proof of Adam and Adagrad Alexandre Défossez Léon Bottou Francis R. Bach Nicolas Usunier 56 143 0 05 Mar 2020