Understanding Batch Normalization

1 June 2018

Papers citing "Understanding Batch Normalization"

50 / 69 papers shown

Title
L-SWAG: Layer-Sample Wise Activation with Gradients information for Zero-Shot NAS on Vision Transformers S. Casarin Sergio Escalera Oswald Lanz 34 0 0 12 May 2025
How to Train Your Metamorphic Deep Neural Network Thomas Sommariva Simone Calderara Angelo Porrello 28 0 0 07 May 2025
Gradient Descent Converges Linearly to Flatter Minima than Gradient Flow in Shallow Linear Networks Pierfrancesco Beneventano Blake Woodworth MLT 36 1 0 15 Jan 2025
Data Generation for Hardware-Friendly Post-Training Quantization Lior Dikstein Ariel Lapid Arnon Netzer H. Habi MQ 154 0 0 29 Oct 2024
Hidden Synergy: $L_1$ Weight Normalization and 1-Path-Norm Regularization Aditya Biswas 41 0 0 29 Apr 2024
K-percent Evaluation for Lifelong RL Golnaz Mesbahi Parham Mohammad Panahi Olya Mastikhina Martha White Adam White CLL OffRL 28 0 0 02 Apr 2024
Boosting Transformer's Robustness and Efficacy in PPG Signal Artifact Detection with Self-Supervised Learning Thanh-Dung Le 31 1 0 02 Jan 2024
Overcoming Recency Bias of Normalization Statistics in Continual Learning: Balance and Adaptation Yilin Lyu Liyuan Wang Xingxing Zhang Zicheng Sun Hang Su Jun Zhu Liping Jing 34 8 0 13 Oct 2023
Uncertainty Quantification for Image-based Traffic Prediction across Cities Alexander Timans Nina Wiedemann Nishant Kumar Ye Hong Martin Raubal 18 1 0 11 Aug 2023
The Implicit Bias of Batch Normalization in Linear Models and Two-layer Linear Convolutional Neural Networks Yuan Cao Difan Zou Yuan-Fang Li Quanquan Gu MLT 29 5 0 20 Jun 2023
On the Weight Dynamics of Deep Normalized Networks Christian H. X. Ali Mehmeti-Göpel Michael Wand 32 1 0 01 Jun 2023
Out-of-distribution Few-shot Learning For Edge Devices without Model Fine-tuning Xinyun Zhang Lanqing Hong OODD 40 0 0 13 Apr 2023
NU-AIR -- A Neuromorphic Urban Aerial Dataset for Detection and Localization of Pedestrians and Vehicles Craig Iaboni Thomas Kelly Pramod Abichandani 21 2 0 18 Feb 2023
IMos: Intent-Driven Full-Body Motion Synthesis for Human-Object Interactions Anindita Ghosh Rishabh Dabral Vladislav Golyanik Christian Theobalt P. Slusallek DiffM 33 86 0 14 Dec 2022
SGD with Large Step Sizes Learns Sparse Features Maksym Andriushchenko Aditya Varre Loucas Pillaud-Vivien Nicolas Flammarion 45 56 0 11 Oct 2022
Dynamical Isometry for Residual Networks Advait Gadhikar R. Burkholz ODL AI4CE 40 2 0 05 Oct 2022
Batch Normalization Explained Randall Balestriero Richard G. Baraniuk AAML 30 16 0 29 Sep 2022
Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction Kaifeng Lyu Zhiyuan Li Sanjeev Arora FAtt 37 69 0 14 Jun 2022
Impact of Learning Rate on Noise Resistant Property of Deep Learning Models Omobayode Fagbohungbe Lijun Qian 24 3 0 08 May 2022
Impact of L1 Batch Normalization on Analog Noise Resistant Property of Deep Learning Models Omobayode Fagbohungbe Lijun Qian 27 0 0 07 May 2022
Relation-guided acoustic scene classification aided with event embeddings Yuanbo Hou Bo Kang Wout Van Hauwermeiren Dick Botteldooren 16 16 0 01 May 2022
Testing Feedforward Neural Networks Training Programs Houssem Ben Braiek Foutse Khomh AAML 11 14 0 01 Apr 2022
Continual Normalization: Rethinking Batch Normalization for Online Continual Learning Quang-Cuong Pham Chenghao Liu S. Hoi BDL OnRL 35 57 0 30 Mar 2022
On the Pitfalls of Batch Normalization for End-to-End Video Learning: A Study on Surgical Workflow Analysis Dominik Rivoir Isabel Funke Stefanie Speidel 24 16 0 15 Mar 2022
projUNN: efficient method for training deep networks with unitary matrices B. Kiani Randall Balestriero Yann LeCun S. Lloyd 41 32 0 10 Mar 2022
Architecture Matters in Continual Learning Seyed Iman Mirzadeh Arslan Chaudhry Dong Yin Timothy Nguyen Razvan Pascanu Dilan Görür Mehrdad Farajtabar OOD KELM 116 58 0 01 Feb 2022
Super-resolution reconstruction of cytoskeleton image based on A-net deep learning network Qian Chen Hao Bai Bingchen Che Tianyun Zhao Ce Zhang Kaige Wang Jintao Bai Wei Zhao 25 3 0 17 Dec 2021
Revisiting Batch Norm Initialization Jim Davis Logan Frank 22 4 0 26 Oct 2021
The Unreasonable Effectiveness of the Final Batch Normalization Layer Veysel Kocaman O. M. Shir T. Baeck 18 1 0 18 Sep 2021
Batch Normalization Preconditioning for Neural Network Training Susanna Lange Kyle E. Helfrich Qiang Ye 27 9 0 02 Aug 2021
Training Energy-Efficient Deep Spiking Neural Networks with Single-Spike Hybrid Input Encoding Gourav Datta Souvik Kundu P. Beerel 40 27 0 26 Jul 2021
A Lightweight ReLU-Based Feature Fusion for Aerial Scene Classification Md. Adnan Arefeen Sumaiya Tabassum Nimi M. Y. S. Uddin Zhu Li 27 10 0 15 Jun 2021
Relating Adversarially Robust Generalization to Flat Minima David Stutz Matthias Hein Bernt Schiele OOD 29 65 0 09 Apr 2021
Delving into Variance Transmission and Normalization: Shift of Average Gradient Makes the Network Collapse YuXiang Liu Jidong Ge Chuanyi Li Jie Gui 21 2 0 22 Mar 2021
On the Validity of Modeling SGD with Stochastic Differential Equations (SDEs) Zhiyuan Li Sadhika Malladi Sanjeev Arora 38 78 0 24 Feb 2021
High-Performance Large-Scale Image Recognition Without Normalization Andrew Brock Soham De Samuel L. Smith Karen Simonyan VLM 223 512 0 11 Feb 2021
Batch Normalization Embeddings for Deep Domain Generalization Mattia Segu A. Tonioni Federico Tombari OOD AI4CE 27 129 0 25 Nov 2020
Studying Robustness of Semantic Segmentation under Domain Shift in cardiac MRI Peter M. Full Fabian Isensee Paul F. Jäger Klaus Maier-Hein OOD 31 43 0 15 Nov 2020
BYOL works even without batch statistics Pierre Harvey Richemond Jean-Bastien Grill Florent Altché Corentin Tallec Florian Strub ... Samuel L. Smith Soham De Razvan Pascanu Bilal Piot Michal Valko SSL 250 114 0 20 Oct 2020
Group Whitening: Balancing Learning Efficiency and Representational Capacity Lei Huang Yi Zhou Li Liu Fan Zhu Ling Shao 20 20 0 28 Sep 2020
Review: Deep Learning in Electron Microscopy Jeffrey M. Ede 31 79 0 17 Sep 2020
DO-Conv: Depthwise Over-parameterized Convolutional Layer Jinming Cao Yangyan Li Mingchao Sun Ying Chen Dani Lischinski Daniel Cohen-Or Baoquan Chen Changhe Tu OOD 31 165 0 22 Jun 2020
New Interpretations of Normalization Methods in Deep Learning Jiacheng Sun Xiangyong Cao Hanwen Liang Weiran Huang Zewei Chen Zhenguo Li 21 34 0 16 Jun 2020
Gradient Centralization: A New Optimization Technique for Deep Neural Networks Hongwei Yong Jianqiang Huang Xiansheng Hua Lei Zhang ODL 24 184 0 03 Apr 2020
Geometric Approaches to Increase the Expressivity of Deep Neural Networks for MR Reconstruction Eunju Cha Gyutaek Oh J. C. Ye 27 11 0 17 Mar 2020
On Feature Normalization and Data Augmentation Boyi Li Felix Wu Ser-Nam Lim Serge J. Belongie Kilian Q. Weinberger 21 134 0 25 Feb 2020
Batch Normalization Biases Residual Blocks Towards the Identity Function in Deep Networks Soham De Samuel L. Smith ODL 14 20 0 24 Feb 2020
The Break-Even Point on Optimization Trajectories of Deep Neural Networks Stanislaw Jastrzebski Maciej Szymczak Stanislav Fort Devansh Arpit Jacek Tabor Kyunghyun Cho Krzysztof J. Geras 44 154 0 21 Feb 2020
Evolution of Image Segmentation using Deep Convolutional Neural Network: A Survey F. Sultana Abu Sufian P. Dutta SSeg 27 249 0 13 Jan 2020
Empirical Studies on the Properties of Linear Regions in Deep Neural Networks Xiao Zhang Dongrui Wu 10 38 0 04 Jan 2020