Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks

25 February 2016

Papers citing "Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks"

50 / 957 papers shown

Title
End-to-End Speech Recognition From the Raw Waveform Neil Zeghidour Nicolas Usunier Gabriel Synnaeve R. Collobert Emmanuel Dupoux 19 84 0 19 Jun 2018
Unsupervised Training for 3D Morphable Model Regression Kyle Genova Forrester Cole Aaron Maschinot Aaron Sarna Daniel Vlasic William T. Freeman CVBM 3DH 33 306 0 15 Jun 2018
Training Faster by Separating Modes of Variation in Batch-normalized Models Mahdi M. Kalayeh M. Shah 27 42 0 07 Jun 2018
AdaGrad stepsizes: Sharp convergence over nonconvex landscapes Rachel A. Ward Xiaoxia Wu Léon Bottou ODL 24 359 0 05 Jun 2018
Inverting Supervised Representations with Autoregressive Neural Density Models C. Nash Nate Kushman Christopher K. I. Williams DRL 8 25 0 01 Jun 2018
Understanding Batch Normalization Johan Bjorck Carla P. Gomes B. Selman Kilian Q. Weinberger 18 593 0 01 Jun 2018
How Does Batch Normalization Help Optimization? Shibani Santurkar Dimitris Tsipras Andrew Ilyas A. Madry ODL 29 1,522 0 29 May 2018
Distributed Weight Consolidation: A Brain Segmentation Case Study Patrick McClure C. Zheng Jakub R. Kaczmarzyk John Rogers-Lee Satrajit S. Ghosh D. Nielson P. Bandettini Francisco Pereira 14 28 0 28 May 2018
Exponential convergence rates for Batch Normalization: The power of length-direction decoupling in non-convex optimization Jonas Köhler Hadi Daneshmand Aurelien Lucchi M. Zhou K. Neymeyr Thomas Hofmann 17 91 0 27 May 2018
Input and Weight Space Smoothing for Semi-supervised Learning Safa Cicek Stefano Soatto 22 6 0 23 May 2018
Learning towards Minimum Hyperspherical Energy Weiyang Liu Rongmei Lin Ziqiang Liu Lixin Liu Zhiding Yu Bo Dai Le Song 24 145 0 23 May 2018
Semi-Supervised Learning with GANs: Revisiting Manifold Regularization Bruno Lecouat Chuan-Sheng Foo Houssam Zenati V. Chandrasekhar GAN 25 29 0 23 May 2018
Approximate Random Dropout Zhuoran Song Ru Wang Dongyu Ru Hongru Huang Zhenghao Peng Hai Zhao Xiaoyao Liang Li Jiang BDL 25 9 0 23 May 2018
Amortized Inference Regularization Rui Shu Hung Bui Shengjia Zhao Mykel J. Kochenderfer Stefano Ermon DRL 13 82 0 23 May 2018
Measuring and regularizing networks in function space Ari S. Benjamin David Rolnick Konrad Paul Kording 21 137 0 21 May 2018
Bilinear Attention Networks Jin-Hwa Kim Jaehyun Jun Byoung-Tak Zhang AIMat 27 867 0 21 May 2018
Batch-Instance Normalization for Adaptively Style-Invariant Neural Networks Hyeonseob Nam Hyo-Eun Kim OOD 16 208 0 21 May 2018
The Best of Both Worlds: Combining Recent Advances in Neural Machine Translation Mengzhao Chen Orhan Firat Ankur Bapna Melvin Johnson Wolfgang Macherey ... Niki Parmar M. Schuster Zhifeng Chen Yonghui Wu Macduff Hughes AIMat 21 0 0 26 Apr 2018
Homocentric Hypersphere Feature Embedding for Person Re-identification Wangmeng Xiang Jianqiang Huang Xianbiao Qi Xiansheng Hua Lei Zhang 11 13 0 24 Apr 2018
Decorrelated Batch Normalization Lei Huang Dawei Yang B. Lang Jia Deng 16 190 0 23 Apr 2018
Stochastic Answer Networks for Natural Language Inference Xiaodong Liu Kevin Duh Jianfeng Gao BDL 13 45 0 21 Apr 2018
Revisiting Small Batch Training for Deep Neural Networks Dominic Masters Carlo Luschi ODL 28 659 0 20 Apr 2018
MaxGain: Regularisation of Neural Networks by Constraining Activation Magnitudes Henry Gouk Bernhard Pfahringer E. Frank M. Cree 16 7 0 16 Apr 2018
A Variational U-Net for Conditional Appearance and Shape Generation Patrick Esser E. Sutter Bjorn Ommer 28 417 0 12 Apr 2018
Regularisation of Neural Networks by Enforcing Lipschitz Continuity Henry Gouk E. Frank Bernhard Pfahringer M. Cree 10 467 0 12 Apr 2018
Neural Autoregressive Flows Chin-Wei Huang David M. Krueger Alexandre Lacoste Aaron Courville DRL AI4CE 22 432 0 03 Apr 2018
Universal Planning Networks A. Srinivas Allan Jabri Pieter Abbeel Sergey Levine Chelsea Finn SSL 25 145 0 02 Apr 2018
Feed-forward Uncertainty Propagation in Belief and Neural Networks Alexander Shekhovtsov B. Flach M. Busta 20 4 0 28 Mar 2018
Normalization of Neural Networks using Analytic Variance Propagation Alexander Shekhovtsov B. Flach 17 6 0 28 Mar 2018
Group Normalization Yuxin Wu Kaiming He 19 3,596 0 22 Mar 2018
VQA-E: Explaining, Elaborating, and Enhancing Your Answers for Visual Questions Qing Li Qingyi Tao Chenyu You Jianfei Cai Jiebo Luo 31 106 0 20 Mar 2018
Deep Co-Training for Semi-Supervised Image Recognition Siyuan Qiao Wei Shen Zhishuai Zhang Bo Wang Alan Yuille 8 444 0 15 Mar 2018
Improving GANs Using Optimal Transport Tim Salimans Han Zhang Alec Radford Dimitris N. Metaxas OT GAN 16 322 0 15 Mar 2018
WNGrad: Learn the Learning Rate in Gradient Descent Xiaoxia Wu Rachel A. Ward Léon Bottou 17 86 0 07 Mar 2018
Norm matters: efficient and accurate normalization schemes in deep networks Elad Hoffer Ron Banner Itay Golan Daniel Soudry OffRL 17 178 0 05 Mar 2018
Accelerating Natural Gradient with Higher-Order Invariance Yang Song Jiaming Song Stefano Ermon 26 21 0 04 Mar 2018
An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling Shaojie Bai J. Zico Kolter V. Koltun DRL 42 4,715 0 04 Mar 2018
Ring loss: Convex Feature Normalization for Face Recognition Yutong Zheng Dipan K. Pal Marios Savvides CVBM 14 198 0 28 Feb 2018
Novelty Detection with GAN M. Kliger S. Fleishman 19 57 0 28 Feb 2018
L1-Norm Batch Normalization for Efficient Training of Deep Neural Networks Shuang Wu Guoqi Li Lei Deng Liu Liu Yuan Xie Luping Shi 12 117 0 27 Feb 2018
Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis Tal Ben-Nun Torsten Hoefler GNN 33 702 0 26 Feb 2018
Content-Based Citation Recommendation Chandra Bhagavatula Sergey Feldman Russell Power Bridger Waleed Ammar 16 149 0 22 Feb 2018
BRUNO: A Deep Recurrent Model for Exchangeable Data I. Korshunova Jonas Degrave Ferenc Huszár Y. Gal Arthur Gretton J. Dambre BDL 24 33 0 21 Feb 2018
Spectral Normalization for Generative Adversarial Networks Takeru Miyato Toshiki Kataoka Masanori Koyama Yuichi Yoshida ODL 23 4,395 0 16 Feb 2018
Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networks Yusuke Tsuzuku Issei Sato Masashi Sugiyama AAML 33 297 0 12 Feb 2018
Batch Kalman Normalization: Towards Training Deep Neural Networks with Micro-Batches Guangrun Wang Jiefeng Peng Ping Luo Xinjiang Wang Liang Lin 29 18 0 09 Feb 2018
Hierarchical Adversarially Learned Inference Mohamed Ishmael Belghazi Sai Rajeswar Olivier Mastropietro Negar Rostamzadeh Jovana Mitrović Aaron Courville GAN BDL 40 29 0 04 Feb 2018
Statistically Motivated Second Order Pooling Kaicheng Yu Mathieu Salzmann 16 42 0 23 Jan 2018
Face Recognition via Centralized Coordinate Learning Xianbiao Qi Lei Zhang CVBM 11 29 0 17 Jan 2018
Understanding the Disharmony between Dropout and Batch Normalization by Variance Shift Xiang Li Shuo Chen Xiaolin Hu Jian Yang 16 309 0 16 Jan 2018