Batch Renormalization: Towards Reducing Minibatch Dependence in Batch-Normalized Models

10 February 2017

Sergey Ioffe

Papers citing "Batch Renormalization: Towards Reducing Minibatch Dependence in Batch-Normalized Models"

41 / 91 papers shown

Title
Batch Normalization Biases Residual Blocks Towards the Identity Function in Deep Networks Soham De Samuel L. Smith ODL 25 20 0 24 Feb 2020
Deep Learning on Image Denoising: An overview Chunwei Tian Lunke Fei Wenxian Zheng Yong-mei Xu W. Zuo Chia-Wen Lin 35 813 0 31 Dec 2019
Big Transfer (BiT): General Visual Representation Learning Alexander Kolesnikov Lucas Beyer Xiaohua Zhai J. Puigcerver Jessica Yung Sylvain Gelly N. Houlsby MQ 103 1,183 0 24 Dec 2019
Latent Replay for Real-Time Continual Learning Lorenzo Pellegrini G. Graffieti Vincenzo Lomonaco Davide Maltoni CLL 28 163 0 02 Dec 2019
Filter Response Normalization Layer: Eliminating Batch Dependence in the Training of Deep Neural Networks Saurabh Singh Shankar Krishnan UQCV 66 125 0 21 Nov 2019
Speaker Adaptive Training using Model Agnostic Meta-Learning Ondˇrej Klejch Joachim Fainberg P. Bell Steve Renals 22 30 0 23 Oct 2019
Root Mean Square Layer Normalization Biao Zhang Rico Sennrich 11 659 0 16 Oct 2019
The Non-IID Data Quagmire of Decentralized Machine Learning Kevin Hsieh Amar Phanishayee O. Mutlu Phillip B. Gibbons 13 558 0 01 Oct 2019
REQ-YOLO: A Resource-Aware, Efficient Quantization Framework for Object Detection on FPGAs Caiwen Ding Shuo Wang Ning Liu Kaidi Xu Yanzhi Wang Yun Liang MQ 16 89 0 29 Sep 2019
Instance Enhancement Batch Normalization: an Adaptive Regulator of Batch Noise Senwei Liang Zhongzhan Huang Mingfu Liang Haizhao Yang 30 57 0 12 Aug 2019
Attentive Normalization Xilai Li Wei Sun Tianfu Wu OOD ViT 28 31 0 04 Aug 2019
Switchable Normalization for Learning-to-Normalize Deep Representation Ping Luo Ruimao Zhang Jiamin Ren Zhanglin Peng Jingyu Li 30 73 0 22 Jul 2019
Rehearsal-Free Continual Learning over Small Non-I.I.D. Batches Vincenzo Lomonaco Davide Maltoni Lorenzo Pellegrini CLL VLM 26 67 0 08 Jul 2019
Offset Calibration for Appearance-Based Gaze Estimation via Gaze Decomposition Zhaokang Chen Bertram E. Shi 26 27 0 11 May 2019
EvalNorm: Estimating Batch Normalization Statistics for Evaluation Saurabh Singh Abhinav Shrivastava 26 51 0 12 Apr 2019
Micro-Batch Training with Batch-Channel Normalization and Weight Standardization Siyuan Qiao Huiyu Wang Chenxi Liu Wei Shen Alan Yuille MQ 32 144 0 25 Mar 2019
Robust and Communication-Efficient Federated Learning from Non-IID Data Felix Sattler Simon Wiedemann K. Müller Wojciech Samek FedML 24 1,331 0 07 Mar 2019
Accelerating Training of Deep Neural Networks with a Standardization Loss Jasmine Collins Johannes Ballé Jonathon Shlens 21 3 0 03 Mar 2019
Segmentation of Roots in Soil with U-Net Abraham George Smith Jens Petersen Raghavendra Selvan C. Rasmussen 19 122 0 28 Feb 2019
Accurate, Data-Efficient, Unconstrained Text Recognition with Convolutional Neural Networks Mohamed Yousef K. Hussain U. S. Mohammed 3DV 23 124 0 31 Dec 2018
Theoretical Analysis of Auto Rate-Tuning by Batch Normalization Sanjeev Arora Zhiyuan Li Kaifeng Lyu 34 130 0 10 Dec 2018
Rethinking ImageNet Pre-training Kaiming He Ross B. Girshick Piotr Dollár VLM SSeg 47 1,076 0 21 Nov 2018
Regularizing by the Variance of the Activations' Sample-Variances Etai Littwin Lior Wolf VLM 17 12 0 21 Nov 2018
Improving Fingerprint Pore Detection with a Small FCN Gabriel Dahia Maurício Pamplona Segundo 20 12 0 14 Nov 2018
Mode Normalization Lucas Deecke Iain Murray Hakan Bilen OOD 29 33 0 12 Oct 2018
Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learning Charles H. Martin Michael W. Mahoney AI4CE 38 191 0 02 Oct 2018
Recent Advances in Object Detection in the Age of Deep Convolutional Neural Networks Shivang Agarwal Jean Ogier du Terrail F. Jurie ObjD 24 123 0 10 Sep 2018
Automatically designing CNN architectures using genetic algorithm for image classification Yizhou Sun Bing Xue Mengjie Zhang Gary G. Yen 3DV 27 603 0 11 Aug 2018
Training Faster by Separating Modes of Variation in Batch-normalized Models Mahdi M. Kalayeh M. Shah 27 42 0 07 Jun 2018
Understanding Batch Normalization Johan Bjorck Carla P. Gomes B. Selman Kilian Q. Weinberger 21 593 0 01 Jun 2018
dhSegment: A generic deep-learning approach for document segmentation S. Oliveira Benoit Seguin F. Kaplan SSeg 25 171 0 27 Apr 2018
Decorrelated Batch Normalization Lei Huang Dawei Yang B. Lang Jia Deng 16 190 0 23 Apr 2018
Revisiting Small Batch Training for Deep Neural Networks Dominic Masters Carlo Luschi ODL 37 659 0 20 Apr 2018
Investigating Generative Adversarial Networks based Speech Dereverberation for Robust Speech Recognition Ke Wang Junbo Zhang Sining Sun Yujun Wang Fei Xiang Lei Xie 20 39 0 27 Mar 2018
Bayesian Uncertainty Estimation for Batch Normalized Deep Networks Mattias Teye Hossein Azizpour Kevin Smith BDL UQCV 28 239 0 18 Feb 2018
The exploding gradient problem demystified - definition, prevalence, impact, origin, tradeoffs, and solutions George Philipp D. Song J. Carbonell ODL 35 46 0 15 Dec 2017
Data Augmentation Generative Adversarial Networks Antreas Antoniou Amos Storkey Harrison Edwards MedIm GAN 65 1,066 0 12 Nov 2017
Riemannian approach to batch normalization Minhyung Cho Jaehyung Lee 29 93 0 27 Sep 2017
Comparison of Batch Normalization and Weight Normalization Algorithms for the Large-scale Image Classification Igor Gitman Boris Ginsburg 8 65 0 24 Sep 2017
Distributed Training Large-Scale Deep Architectures Shang-Xuan Zou Chun-Yen Chen Jui-Lin Wu Chun-Nan Chou Chia-Chin Tsao Kuan-Chieh Tung Ting-Wei Lin Cheng-Lung Sung Edward Y. Chang 26 22 0 10 Aug 2017
A deep convolutional neural network using directional wavelets for low-dose X-ray CT reconstruction Eunhee Kang Junhong Min J. C. Ye OOD MedIm 22 763 0 31 Oct 2016