Understanding the Disharmony between Dropout and Batch Normalization by Variance Shift

16 January 2018

Xiang Li

Papers citing "Understanding the Disharmony between Dropout and Batch Normalization by Variance Shift"

23 / 23 papers shown

Title
SLAB: Efficient Transformers with Simplified Linear Attention and Progressive Re-parameterized Batch Normalization Jialong Guo Xinghao Chen Yehui Tang Yunhe Wang ViT 49 9 0 19 May 2024
Watch Your Head: Assembling Projection Heads to Save the Reliability of Federated Models Jinqian Chen Jihua Zhu Qinghai Zheng Zhongyu Li Zhiqiang Tian FedML 29 3 0 26 Feb 2024
How to Use Dropout Correctly on Residual Networks with Batch Normalization Bum Jun Kim Hyeyeon Choi Hyeonah Jang Donggeon Lee Sang Woo Kim 22 7 0 13 Feb 2023
Efficient brain age prediction from 3D MRI volumes using 2D projections Johan Jönemo Muhammad Usman Akbar Robin Kämpe J. P. Hamilton Anders Eklund 24 5 0 10 Nov 2022
Measuring Overfitting in Convolutional Neural Networks using Adversarial Perturbations and Label Noise Svetlana Pavlitskaya Joël Oswald J. Marius Zöllner NoLa AAML 24 5 0 27 Sep 2022
Batch Normalization Is Blind to the First and Second Derivatives of the Loss Zhanpeng Zhou Wen Shen Huixin Chen Ling Tang Quanshi Zhang 34 2 0 30 May 2022
Testing Feedforward Neural Networks Training Programs Houssem Ben Braiek Foutse Khomh AAML 11 14 0 01 Apr 2022
DualNet: Continual Learning, Fast and Slow Quang-Cuong Pham Chenghao Liu Guosheng Lin CLL 71 43 0 01 Oct 2021
Emerging Relation Network and Task Embedding for Multi-Task Regression Problems Jens Schreiber Bernhard Sick AI4TS 19 12 0 29 Apr 2020
Sequence Model Design for Code Completion in the Modern IDE Gareth Ari Aye Gail E. Kaiser 15 30 0 10 Apr 2020
AL2: Progressive Activation Loss for Learning General Representations in Classification Neural Networks Majed El Helou Frederike Dumbgen Sabine Süsstrunk CLL AI4CE 30 2 0 07 Mar 2020
Identity Recognition in Intelligent Cars with Behavioral Data and LSTM-ResNet Classifier Michael Hammann Maximilian Kraus S. Shafaei Alois C. Knoll 21 3 0 02 Mar 2020
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Zhenzhong Lan Mingda Chen Sebastian Goodman Kevin Gimpel Piyush Sharma Radu Soricut SSL AIMat 97 6,375 0 26 Sep 2019
Semi-Supervised Learning for Cancer Detection of Lymph Node Metastases Amit Kumar Jaiswal Ivan Panshin D. Shulkin Nagender Aneja Samuel Abramov SSL MedIm 20 23 0 23 Jun 2019
S-ConvNet: A Shallow Convolutional Neural Network Architecture for Neuromuscular Activity Recognition Using Instantaneous High-Density Surface EMG Images M. Islam Daniel Massicotte F. Nougarou P. Massicotte Weiping Zhu 21 9 0 08 Jun 2019
Ensemble Model Patching: A Parameter-Efficient Variational Bayesian Neural Network Oscar Chang Yuling Yao David Williams-King Hod Lipson BDL UQCV 32 8 0 23 May 2019
Effective and Efficient Dropout for Deep Convolutional Neural Networks Shaofeng Cai Jinyang Gao Gang Chen Beng Chin Ooi Wei Wang Meihui Zhang BDL 18 53 0 06 Apr 2019
Segmentation of Roots in Soil with U-Net Abraham George Smith Jens Petersen Raghavendra Selvan C. Rasmussen 19 122 0 28 Feb 2019
Mode Normalization Lucas Deecke Iain Murray Hakan Bilen OOD 29 33 0 12 Oct 2018
Towards Understanding Regularization in Batch Normalization Ping Luo Xinjiang Wang Wenqi Shao Zhanglin Peng MLT AI4CE 23 179 0 04 Sep 2018
Adaptive Blending Units: Trainable Activation Functions for Deep Neural Networks L. R. Sütfeld Flemming Brieger Holger Finger S. Füllhase G. Pipa 28 28 0 26 Jun 2018
Understanding Dropout as an Optimization Trick Sangchul Hahn Heeyoul Choi ODL 13 34 0 26 Jun 2018
Backdrop: Stochastic Backpropagation Siavash Golkar Kyle Cranmer 41 2 0 04 Jun 2018