How Does Batch Normalization Help Optimization?

29 May 2018

Papers citing "How Does Batch Normalization Help Optimization?"

48 / 198 papers shown

Title
IC-Network: Efficient Structure for Convolutional Neural Networks Junyi An Fengshan Liu Jian Zhao S. Furao 26 0 0 19 Nov 2019
Understanding and Improving Layer Normalization Jingjing Xu Xu Sun Zhiyuan Zhang Guangxiang Zhao Junyang Lin FAtt 32 342 0 16 Nov 2019
Streaming convolutional neural networks for end-to-end learning with multi-megapixel images H. Pinckaers Bram van Ginneken G. Litjens MedIm 27 94 0 11 Nov 2019
Turbo Autoencoder: Deep learning based channel codes for point-to-point communication channels Yihan Jiang Hyeji Kim Himanshu Asnani Sreeram Kannan Sewoong Oh Pramod Viswanath 30 134 0 08 Nov 2019
Root Mean Square Layer Normalization Biao Zhang Rico Sennrich 19 665 0 16 Oct 2019
Transformers without Tears: Improving the Normalization of Self-Attention Toan Q. Nguyen Julian Salazar 38 224 0 14 Oct 2019
The Non-IID Data Quagmire of Decentralized Machine Learning Kevin Hsieh Amar Phanishayee O. Mutlu Phillip B. Gibbons 13 558 0 01 Oct 2019
Towards Understanding the Transferability of Deep Representations Hong Liu Mingsheng Long Jianmin Wang Michael I. Jordan 30 25 0 26 Sep 2019
Convolutional Neural Networks with Dynamic Regularization Yi Wang Zhen-Peng Bian Junhui Hou Lap-Pui Chau 21 21 0 26 Sep 2019
Generating Accurate Pseudo-labels in Semi-Supervised Learning and Avoiding Overconfident Predictions via Hermite Polynomial Activations Vishnu Suresh Lokhande Songwong Tasneeyapant Abhay Venkatesh Sathya Ravi Vikas Singh 24 29 0 12 Sep 2019
Instance Enhancement Batch Normalization: an Adaptive Regulator of Batch Noise Senwei Liang Zhongzhan Huang Mingfu Liang Haizhao Yang 30 57 0 12 Aug 2019
Attentive Normalization Xilai Li Wei Sun Tianfu Wu OOD ViT 28 31 0 04 Aug 2019
Switchable Normalization for Learning-to-Normalize Deep Representation Ping Luo Ruimao Zhang Jiamin Ren Zhanglin Peng Jingyu Li 30 73 0 22 Jul 2019
Learning to Forget for Meta-Learning Sungyong Baik Seokil Hong Kyoung Mu Lee CLL KELM 22 87 0 13 Jun 2019
Principled Training of Neural Networks with Direct Feedback Alignment Julien Launay Iacopo Poli Florent Krzakala 24 35 0 11 Jun 2019
The Normalization Method for Alleviating Pathological Sharpness in Wide Neural Networks Ryo Karakida S. Akaho S. Amari 27 40 0 07 Jun 2019
An Empirical Study on Hyperparameters and their Interdependence for RL Generalization Xingyou Song Yilun Du Jacob Jackson AI4CE 24 8 0 02 Jun 2019
Why gradient clipping accelerates training: A theoretical justification for adaptivity J.N. Zhang Tianxing He S. Sra Ali Jadbabaie 30 445 0 28 May 2019
Learning to learn via Self-Critique Antreas Antoniou Amos Storkey SSL 23 17 0 24 May 2019
Fine-grained Optimization of Deep Neural Networks Mete Ozay ODL 16 1 0 22 May 2019
Addressing the Loss-Metric Mismatch with Adaptive Loss Alignment Chen Huang Shuangfei Zhai Walter A. Talbott Miguel Angel Bautista Shi Sun Carlos Guestrin J. Susskind 29 75 0 15 May 2019
Deep Neural Networks for Marine Debris Detection in Sonar Images Matias Valdenegro-Toro 27 25 0 13 May 2019
Nested Variational Autoencoder for Topic Modeling on Microtexts with Word Vectors Trung Trinh Tho Quan Trung Mai BDL 19 2 0 01 May 2019
Deep Representation with ReLU Neural Networks Andreas Heinecke W. Hwang 39 0 0 29 Mar 2019
Micro-Batch Training with Batch-Channel Normalization and Weight Standardization Siyuan Qiao Huiyu Wang Chenxi Liu Wei Shen Alan Yuille MQ 32 144 0 25 Mar 2019
Mean-field Analysis of Batch Normalization Ming-Bo Wei J. Stokes D. Schwab MLT 33 8 0 06 Mar 2019
Accelerating Training of Deep Neural Networks with a Standardization Loss Jasmine Collins Johannes Ballé Jonathon Shlens 21 3 0 03 Mar 2019
Towards Robust ResNet: A Small Step but A Giant Leap Jingfeng Zhang Bo Han L. Wynter K. H. Low Mohan Kankanhalli 24 41 0 28 Feb 2019
U-NetPlus: A Modified Encoder-Decoder U-Net Architecture for Semantic and Instance Segmentation of Surgical Instrument S. Hasan Cristian A. Linte MedIm 25 92 0 24 Feb 2019
An Empirical Study of Large-Batch Stochastic Gradient Descent with Structured Covariance Noise Yeming Wen Kevin Luk Maxime Gazeau Guodong Zhang Harris Chan Jimmy Ba ODL 20 22 0 21 Feb 2019
An Investigation into Neural Net Optimization via Hessian Eigenvalue Density Behrooz Ghorbani Shankar Krishnan Ying Xiao ODL 18 317 0 29 Jan 2019
Overfitting Mechanism and Avoidance in Deep Neural Networks Shaeke Salman Xiuwen Liu 14 139 0 19 Jan 2019
Comparing two deep learning sequence-based models for protein-protein interaction prediction Florian Richoux Charlène Servantie C. Borès Stéphane Téletchéa 23 25 0 15 Jan 2019
Theoretical Analysis of Auto Rate-Tuning by Batch Normalization Sanjeev Arora Zhiyuan Li Kaifeng Lyu 34 131 0 10 Dec 2018
Unsupervised domain adaptation for medical imaging segmentation with self-ensembling C. Perone P. Ballester Rodrigo C. Barros Julien Cohen-Adad OOD 33 207 0 14 Nov 2018
Mode Normalization Lucas Deecke Iain Murray Hakan Bilen OOD 29 33 0 12 Oct 2018
Information Geometry of Orthogonal Initializations and Training Piotr A. Sokól Il-Su Park AI4CE 80 16 0 09 Oct 2018
Context-Aware Systems for Sequential Item Recommendation Moin Nadeem D. Stansbury Shane Mooney AI4Ed 19 2 0 21 Sep 2018
A Domain Agnostic Normalization Layer for Unsupervised Adversarial Domain Adaptation Rob Romijnders Panagiotis Meletis Gijs Dubbelman AI4CE 21 27 0 14 Sep 2018
Convolutional Neural Networks for the segmentation of microcalcification in Mammography Imaging Gabriele Valvano Gianmarco Santini N. Martini A. Ripoli C. Iacconi D. Chiappino D. Latta 16 51 0 11 Sep 2018
Towards Understanding Regularization in Batch Normalization Ping Luo Xinjiang Wang Wenqi Shao Zhanglin Peng MLT AI4CE 23 179 0 04 Sep 2018
Ensemble Kalman Inversion: A Derivative-Free Technique For Machine Learning Tasks Nikola B. Kovachki Andrew M. Stuart BDL 44 136 0 10 Aug 2018
Troubling Trends in Machine Learning Scholarship Zachary Chase Lipton Jacob Steinhardt 29 288 0 09 Jul 2018
Restructuring Batch Normalization to Accelerate CNN Training Wonkyung Jung Daejin Jung and Byeongho Kim Sunjung Lee Wonjong Rhee Jung Ho Ahn 24 62 0 04 Jul 2018
Data augmentation instead of explicit regularization Alex Hernández-García Peter König 32 141 0 11 Jun 2018
Whitening and Coloring batch transform for GANs Aliaksandr Siarohin E. Sangineto N. Sebe 22 49 0 01 Jun 2018
Squeeze-and-Excitation Networks Jie Hu Li Shen Samuel Albanie Gang Sun Enhua Wu 84 26,062 0 05 Sep 2017
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 308 2,892 0 15 Sep 2016