Towards Understanding Regularization in Batch Normalization

4 September 2018

Papers citing "Towards Understanding Regularization in Batch Normalization"

33 / 33 papers shown

Title
Overcoming Recency Bias of Normalization Statistics in Continual Learning: Balance and Adaptation Yilin Lyu Liyuan Wang Xingxing Zhang Zicheng Sun Hang Su Jun Zhu Liping Jing 34 8 0 13 Oct 2023
The Implicit Bias of Batch Normalization in Linear Models and Two-layer Linear Convolutional Neural Networks Yuan Cao Difan Zou Yuan-Fang Li Quanquan Gu MLT 37 5 0 20 Jun 2023
Model Agnostic Sample Reweighting for Out-of-Distribution Learning Xiao Zhou Yong Lin Renjie Pi Weizhong Zhang Renzhe Xu Peng Cui Tong Zhang OODD 39 60 0 24 Jan 2023
NoMorelization: Building Normalizer-Free Models from a Sample's Perspective Chang-Shu Liu Yuwen Yang Yue Ding Hongtao Lu 29 2 0 13 Oct 2022
Rethinking Normalization Methods in Federated Learning Zhixu Du Jingwei Sun Ang Li Pin-Yu Chen Jianyi Zhang H. Li Yiran Chen FedML 29 28 0 07 Oct 2022
Understanding and Improving Group Normalization Agus Gunawan Xu Yin Kang Zhang 15 3 0 05 Jul 2022
Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction Kaifeng Lyu Zhiyuan Li Sanjeev Arora FAtt 40 69 0 14 Jun 2022
Neural Collapse: A Review on Modelling Principles and Generalization Vignesh Kothapalli 23 71 0 08 Jun 2022
Batch Normalization Is Blind to the First and Second Derivatives of the Loss Zhanpeng Zhou Wen Shen Huixin Chen Ling Tang Quanshi Zhang 34 2 0 30 May 2022
Noise Regularizes Over-parameterized Rank One Matrix Recovery, Provably Tianyi Liu Yan Li Enlu Zhou Tuo Zhao 38 1 0 07 Feb 2022
Three approaches to facilitate DNN generalization to objects in out-of-distribution orientations and illuminations Akira Sakai Taro Sunagawa Spandan Madan Kanata Suzuki Takashi Katoh Hiromichi Kobashi Hanspeter Pfister Pawan Sinha Xavier Boix Tomotake Sasaki 19 2 0 30 Oct 2021
Revisiting Batch Norm Initialization Jim Davis Logan Frank 22 4 0 26 Oct 2021
Batch Normalization Preconditioning for Neural Network Training Susanna Lange Kyle E. Helfrich Qiang Ye 27 9 0 02 Aug 2021
GemNet: Universal Directional Graph Neural Networks for Molecules Johannes Klicpera Florian Becker Stephan Günnemann AI4CE 39 435 0 02 Jun 2021
"BNN - BN = ?": Training Binary Neural Networks without Batch Normalization Tianlong Chen Zhenyu (Allen) Zhang Xu Ouyang Zechun Liu Zhiqiang Shen Zhangyang Wang MQ 40 36 0 16 Apr 2021
Noisy Gradient Descent Converges to Flat Minima for Nonconvex Matrix Factorization Tianyi Liu Yan Li S. Wei Enlu Zhou T. Zhao 21 13 0 24 Feb 2021
High-Performance Large-Scale Image Recognition Without Normalization Andrew Brock Soham De Samuel L. Smith Karen Simonyan VLM 223 512 0 11 Feb 2021
Variational Capsule Encoder Harish RaviPrakash Syed Muhammad Anwar Ulas Bagci BDL DRL 18 2 0 18 Oct 2020
Group Whitening: Balancing Learning Efficiency and Representational Capacity Lei Huang Yi Zhou Li Liu Fan Zhu Ling Shao 28 20 0 28 Sep 2020
GraphNorm: A Principled Approach to Accelerating Graph Neural Network Training Tianle Cai Shengjie Luo Keyulu Xu Di He Tie-Yan Liu Liwei Wang GNN 32 158 0 07 Sep 2020
New Interpretations of Normalization Methods in Deep Learning Jiacheng Sun Xiangyong Cao Hanwen Liang Weiran Huang Zewei Chen Zhenguo Li 21 34 0 16 Jun 2020
Regularization Methods for Generative Adversarial Networks: An Overview of Recent Studies Minhyeok Lee Junhee Seok GAN 27 25 0 19 May 2020
Gradient Centralization: A New Optimization Technique for Deep Neural Networks Hongwei Yong Jianqiang Huang Xiansheng Hua Lei Zhang ODL 27 184 0 03 Apr 2020
Batch Normalization Biases Residual Blocks Towards the Identity Function in Deep Networks Soham De Samuel L. Smith ODL 14 20 0 24 Feb 2020
The Break-Even Point on Optimization Trajectories of Deep Neural Networks Stanislaw Jastrzebski Maciej Szymczak Stanislav Fort Devansh Arpit Jacek Tabor Kyunghyun Cho Krzysztof J. Geras 50 154 0 21 Feb 2020
A Comprehensive and Modularized Statistical Framework for Gradient Norm Equality in Deep Neural Networks Zhaodong Chen Lei Deng Bangyan Wang Guoqi Li Yuan Xie 35 28 0 01 Jan 2020
Instance Enhancement Batch Normalization: an Adaptive Regulator of Batch Noise Senwei Liang Zhongzhan Huang Mingfu Liang Haizhao Yang 27 57 0 12 Aug 2019
Adaptive Regularization via Residual Smoothing in Deep Learning Optimization Jung-Kyun Cho Junseok Kwon Byung-Woo Hong 28 1 0 23 Jul 2019
Switchable Normalization for Learning-to-Normalize Deep Representation Ping Luo Ruimao Zhang Jiamin Ren Zhanglin Peng Jingyu Li 30 73 0 22 Jul 2019
Deep Neural Networks for Marine Debris Detection in Sonar Images Matias Valdenegro-Toro 27 25 0 13 May 2019
Micro-Batch Training with Batch-Channel Normalization and Weight Standardization Siyuan Qiao Huiyu Wang Chenxi Liu Wei Shen Alan Yuille MQ 32 144 0 25 Mar 2019
Quantifying Generalization in Reinforcement Learning K. Cobbe Oleg Klimov Christopher Hesse Taehoon Kim John Schulman OffRL 12 659 0 06 Dec 2018
Deep Learning for Generic Object Detection: A Survey Li Liu Wanli Ouyang Xiaogang Wang Paul Fieguth Jie Chen Xinwang Liu M. Pietikäinen ObjD VLM OOD 72 2,420 0 06 Sep 2018