Neumann Optimizer: A Practical Optimization Algorithm for Deep Neural
Networks

Neumann Optimizer: A Practical Optimization Algorithm for Deep Neural Networks

8 December 2017

Shankar Krishnan

Rif A. Saurous

Papers citing "Neumann Optimizer: A Practical Optimization Algorithm for Deep Neural Networks"

6 / 6 papers shown

Title
ASDL: A Unified Interface for Gradient Preconditioning in PyTorch Kazuki Osawa Satoki Ishikawa Rio Yokota Shigang Li Torsten Hoefler ODL 46 14 0 08 May 2023
Adaptive scaling of the learning rate by second order automatic differentiation F. Gournay Alban Gossard ODL 31 1 0 26 Oct 2022
M-FAC: Efficient Matrix-Free Approximations of Second-Order Information Elias Frantar Eldar Kurtic Dan Alistarh 13 57 0 07 Jul 2021
Augment your batch: better training with larger batches Elad Hoffer Tal Ben-Nun Itay Hubara Niv Giladi Torsten Hoefler Daniel Soudry ODL 30 72 0 27 Jan 2019
Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis Tal Ben-Nun Torsten Hoefler GNN 33 702 0 26 Feb 2018
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 308 2,892 0 15 Sep 2016