Augment your batch: better training with larger batches

27 January 2019

Papers citing "Augment your batch: better training with larger batches"

35 / 35 papers shown

Title
Patient ADE Risk Prediction through Hierarchical Time-Aware Neural Network Using Claim Codes Jinhe Shi Xiangyu Gao C. Ha Yage Wang Guodong Gao Yi Chen 30 0 0 20 Aug 2020
On the Computational Inefficiency of Large Batch Sizes for Stochastic Gradient Descent Noah Golmant N. Vemuri Z. Yao Vladimir Feinberg A. Gholami Kai Rothauge Michael W. Mahoney Joseph E. Gonzalez 80 73 0 30 Nov 2018
Large-Scale Distributed Second-Order Optimization Using Kronecker-Factored Approximate Curvature for Deep Convolutional Neural Networks Kazuki Osawa Yohei Tsuji Yuichiro Ueno Akira Naruse Rio Yokota Satoshi Matsuoka ODL 73 95 0 29 Nov 2018
Image Classification at Supercomputer Scale Chris Ying Sameer Kumar Dehao Chen Tao Wang Youlong Cheng VLM 51 123 0 16 Nov 2018
Massively Distributed SGD: ImageNet/ResNet-50 Training in a Flash Hiroaki Mikami Hisahiro Suganuma Pongsakorn U-chupala Yoshiki Tanaka Yuichi Kageyama 55 77 0 13 Nov 2018
Measuring the Effects of Data Parallelism on Neural Network Training Christopher J. Shallue Jaehoon Lee J. Antognini J. Mamou J. Ketterling Yao Wang 95 409 0 08 Nov 2018
Highly Scalable Deep Learning Training System with Mixed-Precision: Training ImageNet in Four Minutes Xianyan Jia Shutao Song W. He Yangzihao Wang Haidong Rong ... Li Yu Tiegang Chen Guangxiao Hu Shaoshuai Shi Xiaowen Chu 82 384 0 30 Jul 2018
DARTS: Differentiable Architecture Search Hanxiao Liu Karen Simonyan Yiming Yang 206 4,368 0 24 Jun 2018
AutoAugment: Learning Augmentation Policies from Data E. D. Cubuk Barret Zoph Dandelion Mané Vijay Vasudevan Quoc V. Le 135 1,775 0 24 May 2018
Step Size Matters in Deep Learning Kamil Nar S. Shankar Sastry 40 42 0 22 May 2018
Revisiting Small Batch Training for Deep Neural Networks Dominic Masters Carlo Luschi ODL 71 668 0 20 Apr 2018
Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis Tal Ben-Nun Torsten Hoefler GNN 64 707 0 26 Feb 2018
Horovod: fast and easy distributed deep learning in TensorFlow Alexander Sergeev Mike Del Balso 100 1,222 0 15 Feb 2018
Neumann Optimizer: A Practical Optimization Algorithm for Deep Neural Networks Shankar Krishnan Ying Xiao Rif A. Saurous ODL 41 20 0 08 Dec 2017
Data Augmentation Generative Adversarial Networks Antreas Antoniou Amos Storkey Harrison Edwards MedIm GAN 140 1,073 0 12 Nov 2017
Don't Decay the Learning Rate, Increase the Batch Size Samuel L. Smith Pieter-Jan Kindermans Chris Ying Quoc V. Le ODL 107 996 0 01 Nov 2017
A Bayesian Data Augmentation Approach for Learning Deep Models Toan M. Tran Trung T. Pham G. Carneiro L. Palmer Ian Reid 65 233 0 29 Oct 2017
mixup: Beyond Empirical Risk Minimization Hongyi Zhang Moustapha Cissé Yann N. Dauphin David Lopez-Paz NoLa 301 9,811 0 25 Oct 2017
Deep Learning at 15PF: Supervised and Semi-Supervised Classification for Scientific Data Thorsten Kurth Jian Zhang N. Satish Ioannis Mitliagkas Evan Racah ... J. Deslippe Mikhail Shiryaev Srinivas Sridharan P. Prabhat Pradeep Dubey 50 83 0 17 Aug 2017
Improved Regularization of Convolutional Neural Networks with Cutout Terrance Devries Graham W. Taylor 135 3,775 0 15 Aug 2017
Large Batch Training of Convolutional Networks Yang You Igor Gitman Boris Ginsburg ODL 141 852 0 13 Aug 2017
Regularizing and Optimizing LSTM Language Models Stephen Merity N. Keskar R. Socher 173 1,096 0 07 Aug 2017
Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour Priya Goyal Piotr Dollár Ross B. Girshick P. Noordhuis Lukasz Wesolowski Aapo Kyrola Andrew Tulloch Yangqing Jia Kaiming He 3DH 128 3,685 0 08 Jun 2017
Train longer, generalize better: closing the generalization gap in large batch training of neural networks Elad Hoffer Itay Hubara Daniel Soudry ODL 183 800 0 24 May 2017
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications Andrew G. Howard Menglong Zhu Bo Chen Dmitry Kalenichenko Weijun Wang Tobias Weyand M. Andreetto Hartwig Adam 3DH 1.2K 20,900 0 17 Apr 2017
Exponentially vanishing sub-optimal local minima in multilayer neural networks Daniel Soudry Elad Hoffer 157 97 0 19 Feb 2017
Understanding deep learning requires rethinking generalization Chiyuan Zhang Samy Bengio Moritz Hardt Benjamin Recht Oriol Vinyals HAI 351 4,635 0 10 Nov 2016
RenderGAN: Generating Realistic Labeled Data Leon Sixt Benjamin Wild Tim Landgraf GAN 232 176 0 04 Nov 2016
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 433 2,946 0 15 Sep 2016
Densely Connected Convolutional Networks Gao Huang Zhuang Liu Laurens van der Maaten Kilian Q. Weinberger PINN 3DV 802 36,892 0 25 Aug 2016
Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activations David M. Krueger Tegan Maharaj János Kramár Mohammad Pezeshki Nicolas Ballas Nan Rosemary Ke Anirudh Goyal Yoshua Bengio Aaron Courville C. Pal 84 317 0 03 Jun 2016
Wide Residual Networks Sergey Zagoruyko N. Komodakis 353 8,002 0 23 May 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.2K 194,510 0 10 Dec 2015
Going Deeper with Convolutions Christian Szegedy Wei Liu Yangqing Jia P. Sermanet Scott E. Reed Dragomir Anguelov D. Erhan Vincent Vanhoucke Andrew Rabinovich 494 43,698 0 17 Sep 2014
Very Deep Convolutional Networks for Large-Scale Image Recognition Karen Simonyan Andrew Zisserman FAtt MDE 1.7K 100,529 0 04 Sep 2014