L4: Practical loss-based stepsize adaptation for deep learning

14 February 2018

Papers citing "L4: Practical loss-based stepsize adaptation for deep learning"

17 / 17 papers shown

Title
An Adaptive Stochastic Gradient Method with Non-negative Gauss-Newton Stepsizes Antonio Orvieto Lin Xiao 39 2 0 05 Jul 2024
Stochastic Polyak Step-sizes and Momentum: Convergence Guarantees and Practical Performance Dimitris Oikonomou Nicolas Loizou 55 4 0 06 Jun 2024
Single-Call Stochastic Extragradient Methods for Structured Non-monotone Variational Inequalities: Improved Analysis under Weaker Conditions S. Choudhury Eduard A. Gorbunov Nicolas Loizou 25 13 0 27 Feb 2023
DoG is SGD's Best Friend: A Parameter-Free Dynamic Step Size Schedule Maor Ivgi Oliver Hinder Y. Carmon ODL 26 56 0 08 Feb 2023
QLABGrad: a Hyperparameter-Free and Convergence-Guaranteed Scheme for Deep Learning Minghan Fu Fang-Xiang Wu ODL 22 6 0 01 Feb 2023
Making SGD Parameter-Free Y. Carmon Oliver Hinder 25 41 0 04 May 2022
Amortized Proximal Optimization Juhan Bae Paul Vicol Jeff Z. HaoChen Roger C. Grosse ODL 25 14 0 28 Feb 2022
A Stochastic Bundle Method for Interpolating Networks Alasdair Paren Leonard Berrada Rudra P. K. Poudel M. P. Kumar 24 4 0 29 Jan 2022
Stochastic Mirror Descent: Convergence Analysis and Adaptive Variants via the Mirror Stochastic Polyak Stepsize Ryan DÓrazio Nicolas Loizou I. Laradji Ioannis Mitliagkas 34 30 0 28 Oct 2021
KOALA: A Kalman Optimization Algorithm with Loss Adaptivity A. Davtyan Sepehr Sameni L. Cerkezi Givi Meishvili Adam Bielski Paolo Favaro ODL 53 2 0 07 Jul 2021
How to decay your learning rate Aitor Lewkowycz 36 24 0 23 Mar 2021
Self-Tuning Stochastic Optimization with Curvature-Aware Gradient Filtering Ricky T. Q. Chen Dami Choi Lukas Balles David Duvenaud Philipp Hennig ODL 41 6 0 09 Nov 2020
A straightforward line search approach on the expected empirical loss for stochastic deep learning problems Max Mutschler A. Zell 30 0 0 02 Oct 2020
SGD for Structured Nonconvex Functions: Learning Rates, Minibatching and Interpolation Robert Mansel Gower Othmane Sebbouh Nicolas Loizou 25 74 0 18 Jun 2020
Stochastic Polyak Step-size for SGD: An Adaptive Learning Rate for Fast Convergence Nicolas Loizou Sharan Vaswani I. Laradji Simon Lacoste-Julien 27 181 0 24 Feb 2020
LOSSGRAD: automatic learning rate in gradient descent B. Wójcik Lukasz Maziarka Jacek Tabor ODL 32 4 0 20 Feb 2019
Collaborative Sampling in Generative Adversarial Networks Yuejiang Liu Parth Kothari Alexandre Alahi TTA 28 16 0 02 Feb 2019