v1v2v3v4 (latest)

Understanding Batch Normalization

1 June 2018

Papers citing "Understanding Batch Normalization"

50 / 224 papers shown

Title
P2M: A Processing-in-Pixel-in-Memory Paradigm for Resource-Constrained TinyML Applications Gourav Datta Souvik Kundu Zihan Yin R. T. Lakkireddy Joe Mathai A. Jacob Peter A. Beerel Akhilesh R. Jaiswal 67 41 0 07 Mar 2022
Architecture Matters in Continual Learning Seyed Iman Mirzadeh Arslan Chaudhry Dong Yin Timothy Nguyen Razvan Pascanu Dilan Görür Mehrdad Farajtabar OOD KELM 170 63 0 01 Feb 2022
Rebalancing Batch Normalization for Exemplar-based Class-Incremental Learning Sungmin Cha Sungjun Cho Dasol Hwang Sunwon Hong Moontae Lee Taesup Moon CLL 132 19 0 29 Jan 2022
Super-resolution reconstruction of cytoskeleton image based on A-net deep learning network Qian Chen Hao Bai Bingchen Che Tianyun Zhao Ce Zhang Kaige Wang Jintao Bai Wei Zhao 74 3 0 17 Dec 2021
Training BatchNorm Only in Neural Architecture Search and Beyond Yichen Zhu Jie Du Yuqin Zhu Yi Wang Zhicai Ou Feifei Feng Jian Tang 84 1 0 01 Dec 2021
A Survey on Green Deep Learning Jingjing Xu Wangchunshu Zhou Zhiyi Fu Hao Zhou Lei Li VLM 203 84 0 08 Nov 2021
Revisiting Batch Norm Initialization Jim Davis Logan Frank 66 4 0 26 Oct 2021
Is High Variance Unavoidable in RL? A Case Study in Continuous Control Johan Bjorck Carla P. Gomes Kilian Q. Weinberger 92 23 0 21 Oct 2021
A Riemannian Mean Field Formulation for Two-layer Neural Networks with Batch Normalization Chao Ma Lexing Ying MLT 50 2 0 17 Oct 2021
The Unreasonable Effectiveness of the Final Batch Normalization Layer Veysel Kocaman O. M. Shir T. Baeck 126 1 0 18 Sep 2021
A Multi-view Multi-task Learning Framework for Multi-variate Time Series Forecasting Jinliang Deng Xiusi Chen Renhe Jiang Xuan Song Ivor W. Tsang AI4TS 83 38 0 02 Sep 2021
Batch Normalization Preconditioning for Neural Network Training Susanna Lange Kyle E. Helfrich Qiang Ye 64 9 0 02 Aug 2021
HYPER-SNN: Towards Energy-efficient Quantized Deep Spiking Neural Networks for Hyperspectral Image Classification Gourav Datta Souvik Kundu Akhilesh R. Jaiswal Peter A. Beerel 68 8 0 26 Jul 2021
Training Energy-Efficient Deep Spiking Neural Networks with Single-Spike Hybrid Input Encoding Gourav Datta Souvik Kundu Peter A. Beerel 133 29 0 26 Jul 2021
Towards Low-Latency Energy-Efficient Deep SNNs via Attention-Guided Compression Souvik Kundu Gourav Datta Massoud Pedram Peter A. Beerel 66 14 0 16 Jul 2021
On the Periodic Behavior of Neural Network Training with Batch Normalization and Weight Decay E. Lobacheva M. Kodryan Nadezhda Chirkova A. Malinin Dmitry Vetrov 96 26 0 29 Jun 2021
A Lightweight ReLU-Based Feature Fusion for Aerial Scene Classification Md. Adnan Arefeen Sumaiya Tabassum Nimi M. Y. S. Uddin Zhu Li 106 10 0 15 Jun 2021
Beyond BatchNorm: Towards a Unified Understanding of Normalization in Deep Learning Ekdeep Singh Lubana Robert P. Dick Hidenori Tanaka 80 39 0 10 Jun 2021
Batch Normalization Orthogonalizes Representations in Deep Random Networks Hadi Daneshmand Amir Joudaki Francis R. Bach OOD 68 37 0 07 Jun 2021
Vanishing Curvature and the Power of Adaptive Methods in Randomly Initialized Deep Networks Antonio Orvieto Jonas Köhler Dario Pavllo Thomas Hofmann Aurelien Lucchi ODL 60 5 0 07 Jun 2021
Proxy-Normalizing Activations to Match Batch Normalization while Removing Batch Dependence A. Labatie Dominic Masters Zach Eaton-Rosen Carlo Luschi 135 21 0 07 Jun 2021
Mesh-based graph convolutional neural networks for modeling materials with microstructure A. Frankel Cosmin Safta Coleman Alleman Reese E. Jones 74 15 0 04 Jun 2021
Towards Efficient Full 8-bit Integer DNN Online Training on Resource-limited Devices without Batch Normalization Yukuan Yang Xiaowei Chi Lei Deng Tianyi Yan Feng Gao Guoqi Li MQ 74 6 0 27 May 2021
Noether's Learning Dynamics: Role of Symmetry Breaking in Neural Networks Hidenori Tanaka D. Kunin 113 31 0 06 May 2021
Fitbeat: COVID-19 Estimation based on Wristband Heart Rate Shuo Liu Jing Han Estela Laporta Puyal S. Kontaxis Shaoxiong Sun ... N. Cummins V. A. Narayan Matthew Hotopf Giancarlo Comi Björn Schuller 65 0 0 19 Apr 2021
Relating Adversarially Robust Generalization to Flat Minima David Stutz Matthias Hein Bernt Schiele OOD 105 67 0 09 Apr 2021
Detecting False Data Injection Attacks in Smart Grids with Modeling Errors: A Deep Transfer Learning Based Approach Bowen Xu Fanghong Guo C. Wen Ruilong Deng Wen-an Zhang 49 13 0 09 Apr 2021
Delving into Variance Transmission and Normalization: Shift of Average Gradient Makes the Network Collapse YuXiang Liu Jidong Ge Chuanyi Li Jie Gui 114 2 0 22 Mar 2021
Demystifying Batch Normalization in ReLU Networks: Equivalent Convex Optimization Models and Implicit Regularization Tolga Ergen Arda Sahiner Batu Mehmet Ozturkler John M. Pauly Morteza Mardani Mert Pilanci 117 32 0 02 Mar 2021
On the Validity of Modeling SGD with Stochastic Differential Equations (SDEs) Zhiyuan Li Sadhika Malladi Sanjeev Arora 104 80 0 24 Feb 2021
Sandwich Batch Normalization: A Drop-In Replacement for Feature Distribution Heterogeneity Xinyu Gong Wuyang Chen Tianlong Chen Zhangyang Wang 72 6 0 22 Feb 2021
High-Performance Large-Scale Image Recognition Without Normalization Andrew Brock Soham De Samuel L. Smith Karen Simonyan VLM 328 525 0 11 Feb 2021
Characterizing signal propagation to close the performance gap in unnormalized ResNets Andrew Brock Soham De Samuel L. Smith 199 124 0 21 Jan 2021
Towards Recognizing New Semantic Concepts in New Visual Domains Massimiliano Mancini OOD 141 0 0 16 Dec 2020
Batch Group Normalization Xiao-Yun Zhou Jiacheng Sun Nanyang Ye Xu Lan Qijun Luo Bolin Lai P. Esperança Guang-Zhong Yang Zhenguo Li 153 17 0 04 Dec 2020
Batch Normalization with Enhanced Linear Transformation Yuhui Xu Lingxi Xie Cihang Xie Jieru Mei Siyuan Qiao Wei Shen H. Xiong Alan Yuille 123 0 0 28 Nov 2020
Batch Normalization Embeddings for Deep Domain Generalization Mattia Segu A. Tonioni Federico Tombari OOD AI4CE 134 137 0 25 Nov 2020
Comparing Normalization Methods for Limited Batch Size Segmentation Neural Networks M. Kolarík Radim Burget K. Říha 64 13 0 23 Nov 2020
On tuning deep learning models: a data mining perspective M. Öztürk 21 0 0 19 Nov 2020
Studying Robustness of Semantic Segmentation under Domain Shift in cardiac MRI Peter M. Full Hyunjin Park Paul F. Jäger Klaus Maier-Hein OOD 79 45 0 15 Nov 2020
Data-efficient Alignment of Multimodal Sequences by Aligning Gradient Updates and Internal Feature Distributions Jianan Wang Boyang Albert Li Xiangyu Fan Jing-Hua Lin Yanwei Fu 46 2 0 15 Nov 2020
Neural Network Training Techniques Regularize Optimization Trajectory: An Empirical Study Cheng Chen Junjie Yang Yi Zhou 45 0 0 13 Nov 2020
Improving Model Accuracy for Imbalanced Image Classification Tasks by Adding a Final Batch Normalization Layer: An Empirical Study Veysel Kocaman O. M. Shir Thomas Bäck 65 4 0 12 Nov 2020
Inductive Bias of Gradient Descent for Weight Normalized Smooth Homogeneous Neural Nets Depen Morwani H. G. Ramaswamy 50 3 0 24 Oct 2020
Is Batch Norm unique? An empirical investigation and prescription to emulate the best properties of common normalizers without batch dependence Vinay Rao Jascha Narain Sohl-Dickstein BDL 80 4 0 21 Oct 2020
BYOL works even without batch statistics Pierre Harvey Richemond Jean-Bastien Grill Florent Altché Corentin Tallec Florian Strub ... Samuel L. Smith Soham De Razvan Pascanu Bilal Piot Michal Valko SSL 309 115 0 20 Oct 2020
MimicNorm: Weight Mean and Last BN Layer Mimic the Dynamic of Batch Normalization Wen Fei Wenrui Dai Chenglin Li Junni Zou H. Xiong 39 1 0 19 Oct 2020
Reconciling Modern Deep Learning with Traditional Optimization Analyses: The Intrinsic Learning Rate Zhiyuan Li Kaifeng Lyu Sanjeev Arora 112 75 0 06 Oct 2020
Video Anomaly Detection Using Pre-Trained Deep Convolutional Neural Nets and Context Mining Chongke Wu Sicong Shao Cihan Tunc Salim Hariri 27 19 0 06 Oct 2020
Feature Whitening via Gradient Transformation for Improved Convergence S. Markovich-Golan Barak Battach Amit Bleiweiss 10 1 0 04 Oct 2020