Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks

25 February 2016

Papers citing "Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks"

50 / 957 papers shown

Title
Recurrent Point Review Models K. Cvejoski Ramses J. Sanchez B. Georgiev Christian Bauckhage C. Ojeda KELM BDL 11 2 0 10 Dec 2020
Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning Dynamics D. Kunin Javier Sagastuy-Breña Surya Ganguli Daniel L. K. Yamins Hidenori Tanaka 107 77 0 08 Dec 2020
A Novel Hybrid Framework for Hourly PM2.5 Concentration Forecasting Using CEEMDAN and Deep Temporal Convolutional Neural Network Fuxin Jiang Chengyuan Zhang Shaolong Sun Jingyun Sun AI4Cl 30 4 0 07 Dec 2020
EfficientTTS: An Efficient and High-Quality Text-to-Speech Architecture Chenfeng Miao Shuang Liang Zhencheng Liu Minchuan Chen Jun Ma Shaojun Wang Jing Xiao 22 38 0 07 Dec 2020
Encoding the latent posterior of Bayesian Neural Networks for uncertainty quantification Gianni Franchi Andrei Bursuc Emanuel Aldea Séverine Dubuisson Isabelle Bloch BDL UQCV 22 22 0 04 Dec 2020
Batch Group Normalization Xiao-Yun Zhou Jiacheng Sun Nanyang Ye Xu Lan Qijun Luo Bolin Lai P. Esperança Guang-Zhong Yang Zhenguo Li 24 17 0 04 Dec 2020
Meta-Generating Deep Attentive Metric for Few-shot Classification Lei Zhang Fei Zhou Wei Wei Yanning Zhang VLM 42 28 0 03 Dec 2020
Batch Normalization with Enhanced Linear Transformation Yuhui Xu Lingxi Xie Cihang Xie Jieru Mei Siyuan Qiao Wei Shen H. Xiong Alan Yuille 19 0 0 28 Nov 2020
FBWave: Efficient and Scalable Neural Vocoders for Streaming Text-To-Speech on the Edge Bichen Wu Qing He Peizhao Zhang T. Koehler Kurt Keutzer Peter Vajda 31 6 0 25 Nov 2020
Learning Multiscale Convolutional Dictionaries for Image Reconstruction Tianlin Liu Anadi Chaman David Belius Ivan Dokmanić 25 27 0 25 Nov 2020
Comparing Normalization Methods for Limited Batch Size Segmentation Neural Networks M. Kolarík Radim Burget K. Říha 17 13 0 23 Nov 2020
Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images R. Child BDL VLM 56 339 0 20 Nov 2020
Exploring Simple Siamese Representation Learning Xinlei Chen Kaiming He SSL 58 3,969 0 20 Nov 2020
TFPnP: Tuning-free Plug-and-Play Proximal Algorithm with Applications to Inverse Imaging Problems Kaixuan Wei Angelica Aviles-Rivero Jingwei Liang Ying Fu Hua Huang Carola-Bibiane Schönlieb 27 34 0 18 Nov 2020
Generating Unobserved Alternatives Shichong Peng Ke Li 19 4 0 03 Nov 2020
StyleMelGAN: An Efficient High-Fidelity Adversarial Vocoder with Temporal Adaptive Normalization Ahmed Mustafa N. Pia Guillaume Fuchs 22 72 0 03 Nov 2020
PersGNN: Applying Topological Data Analysis and Geometric Deep Learning to Structure-Based Protein Function Prediction Nicolas Swenson Aditi S. Krishnapriyan A. Buluç Dmitriy Morozov Katherine Yelick 17 16 0 30 Oct 2020
Reducing the Computational Cost of Deep Generative Models with Binary Neural Networks Thomas Bird F. Kingma David Barber SyDa MQ AI4CE 26 9 0 26 Oct 2020
TTS-by-TTS: TTS-driven Data Augmentation for Fast and High-Quality Speech Synthesis Min-Jae Hwang Ryuichi Yamamoto Eunwoo Song Jae-Min Kim 26 31 0 26 Oct 2020
Inductive Bias of Gradient Descent for Weight Normalized Smooth Homogeneous Neural Nets Depen Morwani H. G. Ramaswamy 14 3 0 24 Oct 2020
ResNet or DenseNet? Introducing Dense Shortcuts to ResNet Chaoning Zhang Philipp Benz Dawit Mureja Argaw Seokju Lee Junsik Kim François Rameau Jean-Charles Bazin In So Kweon 26 111 0 23 Oct 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 41 39,551 0 22 Oct 2020
Deep Neural Networks Are Congestion Games: From Loss Landscape to Wardrop Equilibrium and Beyond Nina Vesseron I. Redko Charlotte Laclau 33 5 0 21 Oct 2020
Is Batch Norm unique? An empirical investigation and prescription to emulate the best properties of common normalizers without batch dependence Vinay Rao Jascha Narain Sohl-Dickstein BDL 40 4 0 21 Oct 2020
Real-time Speech Frequency Bandwidth Extension Yunpeng Li Marco Tagliasacchi Oleg Rybakov Victor Ungureanu Dominik Roblek 25 48 0 21 Oct 2020
Smooth activations and reproducibility in deep networks G. Shamir Dong Lin Lorenzo Coviello 22 22 0 20 Oct 2020
Maximum-Entropy Adversarial Data Augmentation for Improved Generalization and Robustness Long Zhao Ting Liu Xi Peng Dimitris N. Metaxas OOD AAML 27 165 0 15 Oct 2020
WeightAlign: Normalizing Activations by Weight Alignment Xiangwei Shi Yun-qiang Li X. Liu Jan van Gemert 19 0 0 14 Oct 2020
A Very Compact Embedded CNN Processor Design Based on Logarithmic Computing Tsung-Ying Lu Hsu-Hsun Chin Hsin-I Wu R. Tsay MQ 27 1 0 13 Oct 2020
HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis Jungil Kong Jaehyeon Kim Jaekyoung Bae 51 1,869 0 12 Oct 2020
Double Forward Propagation for Memorized Batch Normalization Yong Guo Qingyao Wu Chaorui Deng Jian Chen Mingkui Tan 6 28 0 10 Oct 2020
VoiceGrad: Non-Parallel Any-to-Many Voice Conversion with Annealed Langevin Dynamics Hirokazu Kameoka Takuhiro Kaneko Kou Tanaka Nobukatsu Hojo Shogo Seki DiffM 28 21 0 06 Oct 2020
Reconciling Modern Deep Learning with Traditional Optimization Analyses: The Intrinsic Learning Rate Zhiyuan Li Kaifeng Lyu Sanjeev Arora 26 74 0 06 Oct 2020
Self-Supervised Variational Auto-Encoders Ioannis Gatopoulos Jakub M. Tomczak 34 13 0 05 Oct 2020
Neighbourhood Distillation: On the benefits of non end-to-end distillation Laetitia Shao Max Moroz Elad Eban Yair Movshovitz-Attias ODL 18 0 0 02 Oct 2020
Weight and Gradient Centralization in Deep Neural Networks Wolfgang Fuhl Enkelejda Kasneci ODL 13 18 0 02 Oct 2020
VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Models Zhisheng Xiao Karsten Kreis Jan Kautz Arash Vahdat 16 123 0 01 Oct 2020
RG-Flow: A hierarchical and explainable flow model based on renormalization group and sparse prior Hong-Ye Hu Dian Wu Yi-Zhuang You Bruno A. Olshausen Yubei Chen BDL DRL 25 15 0 30 Sep 2020
Normalization Techniques in Training DNNs: Methodology, Analysis and Application Lei Huang Jie Qin Yi Zhou Fan Zhu Li Liu Ling Shao AI4CE 12 255 0 27 Sep 2020
Review: Deep Learning in Electron Microscopy Jeffrey M. Ede 44 79 0 17 Sep 2020
Margin-Based Regularization and Selective Sampling in Deep Neural Networks Berry Weinstein Shai Fine Y. Hel-Or MQ 22 2 0 13 Sep 2020
GraphNorm: A Principled Approach to Accelerating Graph Neural Network Training Tianle Cai Shengjie Luo Keyulu Xu Di He Tie-Yan Liu Liwei Wang GNN 32 160 0 07 Sep 2020
Why Spectral Normalization Stabilizes GANs: Analysis and Improvements Zinan Lin Vyas Sekar Giulia Fanti 12 49 0 06 Sep 2020
SEANet: A Multi-modal Speech Enhancement Network Marco Tagliasacchi Yunpeng Li Karolis Misiunas Dominik Roblek 15 65 0 04 Sep 2020
Training Matters: Unlocking Potentials of Deeper Graph Convolutional Neural Networks Sitao Luan Mingde Zhao Xiao-Wen Chang Doina Precup GNN 18 10 0 20 Aug 2020
A Systematic Survey of Regularization and Normalization in GANs Ziqiang Li Muhammad Usman Rentuo Tao Pengfei Xia Xintian Wu Huanhuan Chen Bin Li 29 50 0 19 Aug 2020
Training Deep Neural Networks Without Batch Normalization D. Gaur Joachim Folz Andreas Dengel ODL 8 10 0 18 Aug 2020
Overcoming Model Bias for Robust Offline Deep Reinforcement Learning Phillip Swazinna Steffen Udluft Thomas Runkler OffRL 23 83 0 12 Aug 2020
The Sockeye 2 Neural Machine Translation Toolkit at AMTA 2020 Tobias Domhan Michael J. Denkowski David Vilar Xing Niu Felix Hieber Kenneth Heafield 16 52 0 11 Aug 2020
Improve Generalization and Robustness of Neural Networks via Weight Scale Shifting Invariant Regularizations Ziquan Liu Yufei Cui Antoni B. Chan 20 13 0 07 Aug 2020