Implicit regularization in Heavy-ball momentum accelerated stochastic
gradient descent

Implicit regularization in Heavy-ball momentum accelerated stochastic gradient descent

2 February 2023

Papers citing "Implicit regularization in Heavy-ball momentum accelerated stochastic gradient descent"

15 / 15 papers shown

Title
Where Do Large Learning Rates Lead Us? Ildus Sadrtdinov M. Kodryan Eduard Pokonechny E. Lobacheva Dmitry Vetrov AI4CE 34 0 0 29 Oct 2024
The AdEMAMix Optimizer: Better, Faster, Older Matteo Pagliardini Pierre Ablin David Grangier ODL 28 8 0 05 Sep 2024
Multiple Instance Verification Xin Xu Eibe Frank G. Holmes 26 0 0 09 Jul 2024
A Margin-based Multiclass Generalization Bound via Geometric Complexity Michael Munn Benoit Dherin Javier Gonzalvo UQCV 40 2 0 28 May 2024
The Impact of Geometric Complexity on Neural Collapse in Transfer Learning Michael Munn Benoit Dherin Javier Gonzalvo AAML 40 1 0 24 May 2024
Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks Hristo Papazov Scott Pesme Nicolas Flammarion 38 5 0 08 Mar 2024
Corridor Geometry in Gradient-Based Optimization Benoit Dherin M. Rosca 32 0 0 13 Feb 2024
Gradient Descent with Polyak's Momentum Finds Flatter Minima via Large Catapults Prin Phunyaphibarn Junghyun Lee Bohan Wang Huishuai Zhang Chulhee Yun 21 0 0 25 Nov 2023
Acceleration and Implicit Regularization in Gaussian Phase Retrieval Tyler Maunu M. Molina-Fructuoso 16 0 0 21 Nov 2023
Implicit biases in multitask and continual learning from a backward error analysis perspective Benoit Dherin 28 3 0 01 Nov 2023
On the Implicit Bias of Adam M. D. Cattaneo Jason M. Klusowski Boris Shigida 31 17 0 31 Aug 2023
The Marginal Value of Momentum for Small Learning Rate SGD Runzhe Wang Sadhika Malladi Tianhao Wang Kaifeng Lyu Zhiyuan Li ODL 44 8 0 27 Jul 2023
Deep Fusion: Efficient Network Training via Pre-trained Initializations Hanna Mazzawi X. Gonzalvo Michael Wunder Sammy Jerome Benoit Dherin AI4CE 33 3 0 20 Jun 2023
Anticorrelated Noise Injection for Improved Generalization Antonio Orvieto Hans Kersting F. Proske Francis R. Bach Aurélien Lucchi 53 44 0 06 Feb 2022
Does Momentum Change the Implicit Regularization on Separable Data? Bohan Wang Qi Meng Huishuai Zhang Ruoyu Sun Wei-Neng Chen Zhirui Ma Tie-Yan Liu 41 15 0 08 Oct 2021