Proxy-Normalizing Activations to Match Batch Normalization while
Removing Batch Dependence

Proxy-Normalizing Activations to Match Batch Normalization while Removing Batch Dependence

7 June 2021

Dominic Masters

Zach Eaton-Rosen

Carlo Luschi

Papers citing "Proxy-Normalizing Activations to Match Batch Normalization while Removing Batch Dependence"

10 / 10 papers shown

Title
Unit Scaling: Out-of-the-Box Low-Precision Training Charlie Blake Douglas Orr Carlo Luschi MQ 24 7 0 20 Mar 2023
NoMorelization: Building Normalizer-Free Models from a Sample's Perspective Chang-Shu Liu Yuwen Yang Yue Ding Hongtao Lu 34 2 0 13 Oct 2022
RevBiFPN: The Fully Reversible Bidirectional Feature Pyramid Network Vitaliy Chiley Vithursan Thangarasa Abhay Gupta Anshul Samar Joel Hestness D. DeCoste 52 8 0 28 Jun 2022
Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction Kaifeng Lyu Zhiyuan Li Sanjeev Arora FAtt 45 70 0 14 Jun 2022
On the Pitfalls of Batch Normalization for End-to-End Video Learning: A Study on Surgical Workflow Analysis Dominik Rivoir Isabel Funke Stefanie Speidel 24 17 0 15 Mar 2022
NanoBatch Privacy: Enabling fast Differentially Private learning on the IPU Edward H. Lee M. M. Krell Alexander Tsyplikhin Victoria Rege E. Colak Kristen W. Yeom FedML 21 0 0 24 Sep 2021
High-Performance Large-Scale Image Recognition Without Normalization Andrew Brock Soham De Samuel L. Smith Karen Simonyan VLM 223 512 0 11 Feb 2021
Is Batch Norm unique? An empirical investigation and prescription to emulate the best properties of common normalizers without batch dependence Vinay Rao Jascha Narain Sohl-Dickstein BDL 40 4 0 21 Oct 2020
Aggregated Residual Transformations for Deep Neural Networks Saining Xie Ross B. Girshick Piotr Dollár Zhuowen Tu Kaiming He 300 10,225 0 16 Nov 2016
A Learned Representation For Artistic Style Vincent Dumoulin Jonathon Shlens M. Kudlur GAN 214 1,156 0 24 Oct 2016