How Does Batch Normalization Help Optimization?

29 May 2018

Papers citing "How Does Batch Normalization Help Optimization?"

50 / 198 papers shown

Title
Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction Kaifeng Lyu Zhiyuan Li Sanjeev Arora FAtt 45 71 0 14 Jun 2022
SmartGD: A GAN-Based Graph Drawing Framework for Diverse Aesthetic Goals Xiaoqi Wang Kevin Yen Yifan Hu Hang Shen 27 4 0 13 Jun 2022
SPD domain-specific batch normalization to crack interpretable unsupervised domain adaptation in EEG Reinmar J. Kobler J. Hirayama Qibin Zhao M. Kawanabe 19 53 0 02 Jun 2022
Batch Normalization Is Blind to the First and Second Derivatives of the Loss Zhanpeng Zhou Wen Shen Huixin Chen Ling Tang Quanshi Zhang 34 2 0 30 May 2022
How to Find Actionable Static Analysis Warnings: A Case Study with FindBugs Rahul Yedida Hong Jin Kang Huy Tu Xueqi Yang David Lo Tim Menzies 40 12 0 21 May 2022
Masterful: A Training Platform for Computer Vision Models S. Wookey Yaoshiang Ho Thomas D. Rikert Juan David Gil Lopez Juan Manuel Munoz Beancur ... Ray Tawil Aaron Sabin Jack Lynch Travis Harper Nikhil Gajendrakumar VLM 23 1 0 21 May 2022
FairNorm: Fair and Fast Graph Neural Network Training Öykü Deniz Köse Yanning Shen AI4CE 21 4 0 20 May 2022
Effect of Batch Normalization on Noise Resistant Property of Deep Learning Models Omobayode Fagbohungbe Lijun Qian 24 10 0 15 May 2022
Impact of L1 Batch Normalization on Analog Noise Resistant Property of Deep Learning Models Omobayode Fagbohungbe Lijun Qian 35 0 0 07 May 2022
On Fragile Features and Batch Normalization in Adversarial Training Nils Philipp Walter David Stutz Bernt Schiele AAML 27 5 0 26 Apr 2022
Receding Neuron Importances for Structured Pruning Mihai Suteu Yike Guo 22 1 0 13 Apr 2022
Online Convolutional Re-parameterization Mu Hu Junyi Feng Jiashen Hua Baisheng Lai Jianqiang Huang Xiaojin Gong Xiansheng Hua 26 26 0 02 Apr 2022
Testing Feedforward Neural Networks Training Programs Houssem Ben Braiek Foutse Khomh AAML 13 14 0 01 Apr 2022
Continual Normalization: Rethinking Batch Normalization for Online Continual Learning Quang Pham Chenghao Liu Guosheng Lin BDL OnRL 38 57 0 30 Mar 2022
Exploiting Low-Rank Tensor-Train Deep Neural Networks Based on Riemannian Gradient Descent With Illustrations of Speech Processing Jun Qi Chao-Han Huck Yang Pin-Yu Chen Javier Tejedor 27 16 0 11 Mar 2022
Ensemble Knowledge Guided Sub-network Search and Fine-tuning for Filter Pruning Seunghyun Lee B. Song 24 8 0 05 Mar 2022
Variational Autoencoders Without the Variation Gregory A. Daly J. Fieldsend G. Tabor 31 2 0 01 Mar 2022
Temporal Efficient Training of Spiking Neural Network via Gradient Re-weighting Shi-Wee Deng Yuhang Li Shanghang Zhang Shi Gu 133 248 0 24 Feb 2022
Multi-Source Unsupervised Domain Adaptation via Pseudo Target Domain Chuan-Xian Ren Yong-Jin Liu Xiwen Zhang Ke-Kun Huang AAML OOD 21 91 0 22 Feb 2022
Diagnosing Batch Normalization in Class Incremental Learning Minghao Zhou Quanziang Wang Jun Shu Qian Zhao Deyu Meng CLL 50 6 0 16 Feb 2022
How Do Vision Transformers Work? Namuk Park Songkuk Kim ViT 47 466 0 14 Feb 2022
DeepStability: A Study of Unstable Numerical Methods and Their Solutions in Deep Learning Eliska Kloberdanz Kyle G. Kloberdanz Wei Le 30 15 0 07 Feb 2022
Interpretability methods of machine learning algorithms with applications in breast cancer diagnosis P. Karatza K. Dalakleidi M. Athanasiou K. Nikita 14 17 0 04 Feb 2022
Architecture Matters in Continual Learning Seyed Iman Mirzadeh Arslan Chaudhry Dong Yin Timothy Nguyen Razvan Pascanu Dilan Görür Mehrdad Farajtabar OOD KELM 118 58 0 01 Feb 2022
Automated Reinforcement Learning (AutoRL): A Survey and Open Problems Jack Parker-Holder Raghunandan Rajan Xingyou Song André Biedenkapp Yingjie Miao ... Vu-Linh Nguyen Roberto Calandra Aleksandra Faust Frank Hutter Marius Lindauer AI4CE 33 100 0 11 Jan 2022
GNN-Geo: A Graph Neural Network-based Fine-grained IP geolocation Framework Shichang Ding Xiangyang Luo Jinwei Wang Xiaoming Fu 35 14 0 18 Dec 2021
Super-resolution reconstruction of cytoskeleton image based on A-net deep learning network Qian Chen Hao Bai Bingchen Che Tianyun Zhao Ce Zhang Kaige Wang Jintao Bai Wei Zhao 30 3 0 17 Dec 2021
Consistent Depth Prediction under Various Illuminations using Dilated Cross Attention Zitian Zhang Chuhua Xian 3DV MDE 43 0 0 15 Dec 2021
TransMorph: Transformer for unsupervised medical image registration Junyu Chen Eric C. Frey Yufan He W. Paul Segars Ye Li Yong Du ViT MedIm 41 304 0 19 Nov 2021
Curriculum Learning for Vision-and-Language Navigation Jiwen Zhang Zhongyu Wei Jianqing Fan J. Peng LM&Ro 26 21 0 14 Nov 2021
Blending Anti-Aliasing into Vision Transformer Shengju Qian Hao Shao Yi Zhu Mu Li Jiaya Jia 28 20 0 28 Oct 2021
Revisiting Batch Norm Initialization Jim Davis Logan Frank 22 4 0 26 Oct 2021
Training Deep Neural Networks with Joint Quantization and Pruning of Weights and Activations Xinyu Zhang Ian Colbert Ken Kreutz-Delgado Srinjoy Das MQ 32 11 0 15 Oct 2021
A Loss Curvature Perspective on Training Instability in Deep Learning Justin Gilmer Behrooz Ghorbani Ankush Garg Sneha Kudugunta Behnam Neyshabur David E. Cardoze George E. Dahl Zachary Nado Orhan Firat ODL 36 35 0 08 Oct 2021
An Expert System for Redesigning Software for Cloud Applications Rahul Yedida R. Krishna A. Kalia Tim Menzies Jin Xiao M. Vukovic 16 4 0 29 Sep 2021
The Unreasonable Effectiveness of the Final Batch Normalization Layer Veysel Kocaman O. M. Shir T. Baeck 18 1 0 18 Sep 2021
Improving Contrastive Learning by Visualizing Feature Transformation Rui Zhu Bingchen Zhao Jingen Liu Zhenglong Sun Chen Chen SSL 99 78 0 06 Aug 2021
Unsupervised Domain Adaptation for Retinal Vessel Segmentation with Adversarial Learning and Transfer Normalization Wei Feng Lie Ju Lin Wang Kaimin Song Xin Wang Xin Zhao Qingyi Tao Z. Ge OOD MedIm 18 4 0 04 Aug 2021
Batch Normalization Preconditioning for Neural Network Training Susanna Lange Kyle E. Helfrich Qiang Ye 27 9 0 02 Aug 2021
SimROD: A Simple Adaptation Method for Robust Object Detection Rindranirina Ramamonjison Amin Banitalebi-Dehkordi Xinyu Kang Xiaolong Bai Yong Zhang ObjD TTA 26 53 0 28 Jul 2021
DeltaCharger: Charging Robot with Inverted Delta Mechanism and CNN-driven High Fidelity Tactile Perception for Precise 3D Positioning Iaroslav Okunevich Daria Trinitatova Pavel Kopanev Dzmitry Tsetserukou 25 11 0 22 Jul 2021
Unsupervised Model Drift Estimation with Batch Normalization Statistics for Dataset Shift Detection and Model Selection Won-Jo Lee Seokhyun Byun Jooeun Kim Minje Park Kirill Chechil AI4TS 21 2 0 01 Jul 2021
The Values Encoded in Machine Learning Research Abeba Birhane Pratyusha Kalluri Dallas Card William Agnew Ravit Dotan Michelle Bao 41 274 0 29 Jun 2021
GemNet: Universal Directional Graph Neural Networks for Molecules Johannes Klicpera Florian Becker Stephan Günnemann AI4CE 39 438 0 02 Jun 2021
Spectral Normalisation for Deep Reinforcement Learning: an Optimisation Perspective Florin Gogianu Tudor Berariu Mihaela Rosca Claudia Clopath L. Buşoniu Razvan Pascanu 24 53 0 11 May 2021
"BNN - BN = ?": Training Binary Neural Networks without Batch Normalization Tianlong Chen Zhenyu Zhang Xu Ouyang Zechun Liu Zhiqiang Shen Zhangyang Wang MQ 46 36 0 16 Apr 2021
Deep Recursive Embedding for High-Dimensional Data Zixia Zhou Yuanyuan Wang B. Lelieveldt Qian Tao 24 7 0 12 Apr 2021
Disentangled Contrastive Learning for Learning Robust Textual Representations Xiang Chen Xin Xie Zhen Bi Hongbin Ye Shumin Deng Ningyu Zhang Huajun Chen 33 5 0 11 Apr 2021
Relating Adversarially Robust Generalization to Flat Minima David Stutz Matthias Hein Bernt Schiele OOD 38 65 0 09 Apr 2021
Delving into Variance Transmission and Normalization: Shift of Average Gradient Makes the Network Collapse YuXiang Liu Jidong Ge Chuanyi Li Jie Gui 21 2 0 22 Mar 2021