Towards Understanding Regularization in Batch Normalization

4 September 2018

Papers citing "Towards Understanding Regularization in Batch Normalization"

29 / 29 papers shown

Title
Overcoming Recency Bias of Normalization Statistics in Continual Learning: Balance and Adaptation Yilin Lyu Liyuan Wang Xingxing Zhang Zicheng Sun Hang Su Jun Zhu Liping Jing 34 8 0 13 Oct 2023
The Implicit Bias of Batch Normalization in Linear Models and Two-layer Linear Convolutional Neural Networks Yuan Cao Difan Zou Yuan-Fang Li Quanquan Gu MLT 29 5 0 20 Jun 2023
Model Agnostic Sample Reweighting for Out-of-Distribution Learning Xiao Zhou Yong Lin Renjie Pi Weizhong Zhang Renzhe Xu Peng Cui Tong Zhang OODD 33 60 0 24 Jan 2023
Understanding and Improving Group Normalization Agus Gunawan Xu Yin Kang Zhang 13 3 0 05 Jul 2022
Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction Kaifeng Lyu Zhiyuan Li Sanjeev Arora FAtt 37 69 0 14 Jun 2022
Neural Collapse: A Review on Modelling Principles and Generalization Vignesh Kothapalli 21 71 0 08 Jun 2022
Batch Normalization Is Blind to the First and Second Derivatives of the Loss Zhanpeng Zhou Wen Shen Huixin Chen Ling Tang Quanshi Zhang 34 2 0 30 May 2022
Noise Regularizes Over-parameterized Rank One Matrix Recovery, Provably Tianyi Liu Yan Li Enlu Zhou Tuo Zhao 38 1 0 07 Feb 2022
Revisiting Batch Norm Initialization Jim Davis Logan Frank 22 4 0 26 Oct 2021
Batch Normalization Preconditioning for Neural Network Training Susanna Lange Kyle E. Helfrich Qiang Ye 27 9 0 02 Aug 2021
GemNet: Universal Directional Graph Neural Networks for Molecules Johannes Klicpera Florian Becker Stephan Günnemann AI4CE 24 434 0 02 Jun 2021
"BNN - BN = ?": Training Binary Neural Networks without Batch Normalization Tianlong Chen Zhenyu (Allen) Zhang Xu Ouyang Zechun Liu Zhiqiang Shen Zhangyang Wang MQ 37 36 0 16 Apr 2021
Noisy Gradient Descent Converges to Flat Minima for Nonconvex Matrix Factorization Tianyi Liu Yan Li S. Wei Enlu Zhou T. Zhao 19 13 0 24 Feb 2021
High-Performance Large-Scale Image Recognition Without Normalization Andrew Brock Soham De Samuel L. Smith Karen Simonyan VLM 223 512 0 11 Feb 2021
Variational Capsule Encoder Harish RaviPrakash Syed Muhammad Anwar Ulas Bagci BDL DRL 13 2 0 18 Oct 2020
GraphNorm: A Principled Approach to Accelerating Graph Neural Network Training Tianle Cai Shengjie Luo Keyulu Xu Di He Tie-Yan Liu Liwei Wang GNN 23 158 0 07 Sep 2020
New Interpretations of Normalization Methods in Deep Learning Jiacheng Sun Xiangyong Cao Hanwen Liang Weiran Huang Zewei Chen Zhenguo Li 18 34 0 16 Jun 2020
Regularization Methods for Generative Adversarial Networks: An Overview of Recent Studies Minhyeok Lee Junhee Seok GAN 27 25 0 19 May 2020
Gradient Centralization: A New Optimization Technique for Deep Neural Networks Hongwei Yong Jianqiang Huang Xiansheng Hua Lei Zhang ODL 16 184 0 03 Apr 2020
Batch Normalization Biases Residual Blocks Towards the Identity Function in Deep Networks Soham De Samuel L. Smith ODL 14 20 0 24 Feb 2020
The Break-Even Point on Optimization Trajectories of Deep Neural Networks Stanislaw Jastrzebski Maciej Szymczak Stanislav Fort Devansh Arpit Jacek Tabor Kyunghyun Cho Krzysztof J. Geras 42 154 0 21 Feb 2020
A Comprehensive and Modularized Statistical Framework for Gradient Norm Equality in Deep Neural Networks Zhaodong Chen Lei Deng Bangyan Wang Guoqi Li Yuan Xie 29 28 0 01 Jan 2020
Instance Enhancement Batch Normalization: an Adaptive Regulator of Batch Noise Senwei Liang Zhongzhan Huang Mingfu Liang Haizhao Yang 25 57 0 12 Aug 2019
Adaptive Regularization via Residual Smoothing in Deep Learning Optimization Jung-Kyun Cho Junseok Kwon Byung-Woo Hong 26 1 0 23 Jul 2019
Switchable Normalization for Learning-to-Normalize Deep Representation Ping Luo Ruimao Zhang Jiamin Ren Zhanglin Peng Jingyu Li 28 73 0 22 Jul 2019
Deep Neural Networks for Marine Debris Detection in Sonar Images Matias Valdenegro-Toro 27 25 0 13 May 2019
Micro-Batch Training with Batch-Channel Normalization and Weight Standardization Siyuan Qiao Huiyu Wang Chenxi Liu Wei Shen Alan Yuille MQ 29 144 0 25 Mar 2019
Quantifying Generalization in Reinforcement Learning K. Cobbe Oleg Klimov Christopher Hesse Taehoon Kim John Schulman OffRL 10 658 0 06 Dec 2018
Deep Learning for Generic Object Detection: A Survey Li Liu Wanli Ouyang Xiaogang Wang Paul Fieguth Jie Chen Xinwang Liu M. Pietikäinen ObjD VLM OOD 70 2,419 0 06 Sep 2018