Adaptive Gradient Methods Converge Faster with Over-Parameterization (but you should do a line-search)

11 June 2020

Papers citing "Adaptive Gradient Methods Converge Faster with Over-Parameterization (but you should do a line-search)"

9 / 9 papers shown

Title
Convergence Conditions for Stochastic Line Search Based Optimization of Over-parametrized Models Matteo Lapucci Davide Pucci 37 1 0 06 Aug 2024
Faster Convergence of Stochastic Accelerated Gradient Descent under Interpolation Aaron Mishkin Mert Pilanci Mark Schmidt 71 1 0 03 Apr 2024
Faster Convergence for Transformer Fine-tuning with Line Search Methods Philip Kenneweg Leonardo Galli Tristan Kenneweg Barbara Hammer ODL 51 2 0 27 Mar 2024
Nest Your Adaptive Algorithm for Parameter-Agnostic Nonconvex Minimax Optimization Junchi Yang Xiang Li Niao He ODL 45 22 0 01 Jun 2022
Towards Noise-adaptive, Problem-adaptive (Accelerated) Stochastic Gradient Descent Sharan Vaswani Benjamin Dubois-Taine Reza Babanezhad 56 11 0 21 Oct 2021
SVRG Meets AdaGrad: Painless Variance Reduction Benjamin Dubois-Taine Sharan Vaswani Reza Babanezhad Mark Schmidt Simon Lacoste-Julien 23 18 0 18 Feb 2021
A Weakly Supervised Consistency-based Learning Method for COVID-19 Segmentation in CT Images I. Laradji Pau Rodríguez López Oscar Manas Keegan Lensink M. Law Lironne Kurzman William Parker David Vazquez Derek Nowrouzezahrai 26 84 0 04 Jul 2020
A new regret analysis for Adam-type algorithms Ahmet Alacaoglu Yura Malitsky P. Mertikopoulos V. Cevher ODL 48 42 0 21 Mar 2020
L4: Practical loss-based stepsize adaptation for deep learning Michal Rolínek Georg Martius ODL 51 63 0 14 Feb 2018