On Empirical Comparisons of Optimizers for Deep Learning

v1v2v3 (latest)

On Empirical Comparisons of Optimizers for Deep Learning

11 October 2019

Christopher J. Shallue

Chris J. Maddison

ArXiv (abs)PDF HTML

Papers citing "On Empirical Comparisons of Optimizers for Deep Learning"

5 / 105 papers shown

Title
FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence Kihyuk Sohn David Berthelot Chun-Liang Li Zizhao Zhang Nicholas Carlini E. D. Cubuk Alexey Kurakin Han Zhang Colin Raffel AAML 173 3,603 0 21 Jan 2020
Gradient descent with momentum --- to accelerate or to super-accelerate? Goran Nakerst John Brennan M. Haque ODL 33 15 0 17 Jan 2020
Individual predictions matter: Assessing the effect of data ordering in training fine-tuned CNNs for medical imaging J. Zech Jessica Zosa Forde Michael L. Littman 38 5 0 08 Dec 2019
Optimizer Benchmarking Needs to Account for Hyperparameter Tuning Prabhu Teja Sivaprasad Florian Mai Thijs Vogels Martin Jaggi François Fleuret 83 12 0 25 Oct 2019
Demon: Improved Neural Network Training with Momentum Decay John Chen Cameron R. Wolfe Zhaoqi Li Anastasios Kyrillidis ODL 106 15 0 11 Oct 2019