DeepOBS: A Deep Learning Optimizer Benchmark Suite

DeepOBS: A Deep Learning Optimizer Benchmark Suite

13 March 2019

Frank Schneider

Papers citing "DeepOBS: A Deep Learning Optimizer Benchmark Suite"

12 / 12 papers shown

Title
Training neural networks faster with minimal tuning using pre-computed lists of hyperparameters for NAdamW Sourabh Medapati Priya Kasimbeg Shankar Krishnan Naman Agarwal George E. Dahl 64 0 0 06 Mar 2025
Learning Versatile Optimizers on a Compute Diet A. Moudgil Boris Knyazev Guillaume Lajoie Eugene Belilovsky 226 0 0 22 Jan 2025
Towards Better Open-Ended Text Generation: A Multicriteria Evaluation Framework Esteban Garces Arias Hannah Blocher Julian Rodemann Meimingwei Li Christian Heumann Matthias Aßenmacher 28 1 0 24 Oct 2024
Debiasing Mini-Batch Quadratics for Applications in Deep Learning Lukas Tatzel Bálint Mucsányi Osane Hackel Philipp Hennig 51 0 0 18 Oct 2024
Deconstructing What Makes a Good Optimizer for Language Models Rosie Zhao Depen Morwani David Brandfonbrener Nikhil Vyas Sham Kakade 52 17 0 10 Jul 2024
HesScale: Scalable Computation of Hessian Diagonals Mohamed Elsayed A. R. Mahmood 22 8 0 20 Oct 2022
A Stochastic Bundle Method for Interpolating Networks Alasdair Paren Leonard Berrada Rudra P. K. Poudel M. P. Kumar 24 4 0 29 Jan 2022
Large-Scale Deep Learning Optimizations: A Comprehensive Survey Xiaoxin He Fuzhao Xue Xiaozhe Ren Yang You 32 14 0 01 Nov 2021
Self-Tuning Stochastic Optimization with Curvature-Aware Gradient Filtering Ricky T. Q. Chen Dami Choi Lukas Balles David Duvenaud Philipp Hennig ODL 44 6 0 09 Nov 2020
Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers Robin M. Schmidt Frank Schneider Philipp Hennig ODL 42 162 0 03 Jul 2020
On Empirical Comparisons of Optimizers for Deep Learning Dami Choi Christopher J. Shallue Zachary Nado Jaehoon Lee Chris J. Maddison George E. Dahl 18 256 0 11 Oct 2019
L4: Practical loss-based stepsize adaptation for deep learning Michal Rolínek Georg Martius ODL 46 63 0 14 Feb 2018