Revisiting Small Batch Training for Deep Neural Networks

Revisiting Small Batch Training for Deep Neural Networks

20 April 2018

Dominic Masters

Carlo Luschi

Papers citing "Revisiting Small Batch Training for Deep Neural Networks"

17 / 167 papers shown

Title
Toward Understanding the Impact of Staleness in Distributed Machine Learning Wei-Ming Dai Yi Zhou Nanqing Dong Huan Zhang Eric P. Xing 22 80 0 08 Oct 2018
Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learning Charles H. Martin Michael W. Mahoney AI4CE 35 191 0 02 Oct 2018
Improving Optimization Bounds using Machine Learning: Decision Diagrams meet Deep Reinforcement Learning Quentin Cappart Emmanuel Goutierre David Bergman Louis-Martin Rousseau AI4CE 15 55 0 10 Sep 2018
Don't Use Large Mini-Batches, Use Local SGD Tao R. Lin Sebastian U. Stich Kumar Kshitij Patel Martin Jaggi 57 429 0 22 Aug 2018
Differentially-Private "Draw and Discard" Machine Learning Vasyl Pihur Aleksandra Korolova Frederick Liu Subhash Sankuratripati M. Yung Dachuan Huang Ruogu Zeng FedML 24 39 0 11 Jul 2018
Pushing the boundaries of parallel Deep Learning -- A practical approach Paolo Viviani M. Drocco Marco Aldinucci OOD 20 0 0 25 Jun 2018
Faster SGD training by minibatch persistency M. Fischetti Iacopo Mandatelli Domenico Salvagnin 11 5 0 19 Jun 2018
Full deep neural network training on a pruned weight budget Maximilian Golub G. Lemieux Mieszko Lis 33 28 0 11 Jun 2018
The Effect of Network Width on the Performance of Large-batch Training Lingjiao Chen Hongyi Wang Jinman Zhao Dimitris Papailiopoulos Paraschos Koutris 16 22 0 11 Jun 2018
Layer rotation: a surprisingly powerful indicator of generalization in deep networks? Simon Carbonnelle Christophe De Vleeschouwer MLT 11 1 0 05 Jun 2018
Analysis of DAWNBench, a Time-to-Accuracy Machine Learning Performance Benchmark Cody Coleman Daniel Kang Deepak Narayanan Luigi Nardi Tian Zhao Jian Zhang Peter Bailis K. Olukotun Christopher Ré Matei A. Zaharia 13 117 0 04 Jun 2018
Backdrop: Stochastic Backpropagation Siavash Golkar Kyle Cranmer 36 2 0 04 Jun 2018
Understanding Batch Normalization Johan Bjorck Carla P. Gomes B. Selman Kilian Q. Weinberger 18 593 0 01 Jun 2018
Amortized Inference Regularization Rui Shu Hung Bui Shengjia Zhao Mykel J. Kochenderfer Stefano Ermon DRL 13 82 0 23 May 2018
Combo Loss: Handling Input and Output Imbalance in Multi-Organ Segmentation Saeid Asgari Taghanaki Yefeng Zheng S. Kevin Zhou Bogdan Georgescu Puneet Sharma Daguang Xu Dorin Comaniciu Ghassan Hamarneh 18 328 0 08 May 2018
Deep supervised learning using local errors Hesham Mostafa V. Ramesh Gert Cauwenberghs 25 113 0 17 Nov 2017
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 308 2,890 0 15 Sep 2016