All you need is a good init

19 November 2015

Papers citing "All you need is a good init"

42 / 92 papers shown

Title
On Graph Classification Networks, Datasets and Baselines Enxhell Luzhnica Ben Day Pietro Lió GNN 18 19 0 12 May 2019
On the security relevance of weights in deep learning Kathrin Grosse T. A. Trost Marius Mosbach Michael Backes Dietrich Klakow AAML 32 6 0 08 Feb 2019
Parameter Re-Initialization through Cyclical Batch Size Schedules Norman Mu Z. Yao A. Gholami Kurt Keutzer Michael W. Mahoney ODL 27 8 0 04 Dec 2018
Self-Referenced Deep Learning Xu Lan Xiatian Zhu S. Gong 21 23 0 19 Nov 2018
Good Initializations of Variational Bayes for Deep Models Simone Rossi Pietro Michiardi Maurizio Filippone BDL 17 21 0 18 Oct 2018
Removing the Feature Correlation Effect of Multiplicative Noise Zijun Zhang Yining Zhang Zongpeng Li 13 8 0 19 Sep 2018
Neural Network Encapsulation Hongyang Li Xiaoyang Guo Bo Dai Wanli Ouyang Xiaogang Wang 21 51 0 11 Aug 2018
Ensemble Kalman Inversion: A Derivative-Free Technique For Machine Learning Tasks Nikola B. Kovachki Andrew M. Stuart BDL 42 136 0 10 Aug 2018
Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks Lechao Xiao Yasaman Bahri Jascha Narain Sohl-Dickstein S. Schoenholz Jeffrey Pennington 220 348 0 14 Jun 2018
On Tighter Generalization Bound for Deep Neural Networks: CNNs, ResNets, and Beyond Xingguo Li Junwei Lu Zhaoran Wang Jarvis D. Haupt T. Zhao 25 78 0 13 Jun 2018
Understanding Batch Normalization Johan Bjorck Carla P. Gomes B. Selman Kilian Q. Weinberger 15 593 0 01 Jun 2018
Learning the Localization Function: Machine Learning Approach to Fingerprinting Localization Linchen Xiao Arash Behboodi R. Mathar 12 12 0 21 Mar 2018
The History Began from AlexNet: A Comprehensive Survey on Deep Learning Approaches Md. Zahangir Alom T. Taha C. Yakopcic Stefan Westberg P. Sidike Mst Shamima Nasrin B. Van Essen A. Awwal V. Asari VLM 29 873 0 03 Mar 2018
Diversity and degrees of freedom in regression ensembles Henry W. J. Reeve Gavin Brown UQCV 15 22 0 01 Mar 2018
The Emergence of Spectral Universality in Deep Networks Jeffrey Pennington S. Schoenholz Surya Ganguli 11 170 0 27 Feb 2018
Deep Metric Learning with BIER: Boosting Independent Embeddings Robustly M. Opitz Georg Waltner Horst Possegger Horst Bischof FedML OOD 30 166 0 15 Jan 2018
Improved Inception-Residual Convolutional Neural Network for Object Recognition Md. Zahangir Alom Mahmudul Hasan C. Yakopcic T. Taha V. Asari 46 116 0 28 Dec 2017
Learning Sight from Sound: Ambient Sound Provides Supervision for Visual Learning Andrew Owens Jiajun Wu Josh H. McDermott William T. Freeman Antonio Torralba SSL 30 177 0 20 Dec 2017
The exploding gradient problem demystified - definition, prevalence, impact, origin, tradeoffs, and solutions George Philipp D. Song J. Carbonell ODL 27 46 0 15 Dec 2017
Invariance of Weight Distributions in Rectified MLPs Russell Tsuchida Farbod Roosta-Khorasani M. Gallagher MLT 24 35 0 24 Nov 2017
Knowledge Projection for Deep Neural Networks Zhi Zhang G. Ning Zhihai He 38 15 0 26 Oct 2017
A systematic study of the class imbalance problem in convolutional neural networks Mateusz Buda A. Maki Maciej Mazurowski 27 2,317 0 15 Oct 2017
High-dimensional dynamics of generalization error in neural networks Madhu S. Advani Andrew M. Saxe AI4CE 47 464 0 10 Oct 2017
Pyramidal RoR for Image Classification Ke Zhang Liru Guo Ce Gao Zhenbing Zhao 36 20 0 01 Oct 2017
Generalizing the Convolution Operator in Convolutional Neural Networks Kamaledin Ghiasi-Shirazi 13 38 0 14 Jul 2017
Transfer entropy-based feedback improves performance in artificial neural networks S. Herzog Christian Tetzlaff F. Worgotter 11 7 0 13 Jun 2017
On weight initialization in deep neural networks S. Kumar ODL 13 223 0 28 Apr 2017
Inception Recurrent Convolutional Neural Network for Object Recognition Md. Zahangir Alom Mahmudul Hasan C. Yakopcic T. Taha 31 86 0 25 Apr 2017
Dynamic Edge-Conditioned Filters in Convolutional Neural Networks on Graphs M. Simonovsky N. Komodakis GNN 19 1,220 0 10 Apr 2017
Colorization as a Proxy Task for Visual Understanding Gustav Larsson Michael Maire Gregory Shakhnarovich SSL 42 493 0 11 Mar 2017
All You Need is Beyond a Good Init: Exploring Better Solution for Training Extremely Deep Convolutional Neural Networks with Orthonormality and Modulation Di Xie Jiang Xiong Shiliang Pu 19 181 0 06 Mar 2017
Regularizing CNNs with Locally Constrained Decorrelations Pau Rodríguez López Jordi Gonzalez Guillem Cucurull J. M. Gonfaus F. X. Roca 24 133 0 07 Nov 2016
X-CNN: Cross-modal Convolutional Neural Networks for Sparse Datasets Petar Velickovic Duo Wang Nicholas D. Lane Pietro Lió 16 31 0 01 Oct 2016
Lets keep it simple, Using simple architectures to outperform deeper and more complex architectures S. H. HasanPour Mohammad Rouhani Mohsen Fayyaz Mohammad Sabokrou 18 118 0 22 Aug 2016
Towards Evaluating the Robustness of Neural Networks Nicholas Carlini D. Wagner OOD AAML 22 8,436 0 16 Aug 2016
Residual Networks of Residual Networks: Multilevel Residual Networks Ke Zhang Miao Sun T. Han Xingfang Yuan Liru Guo Tao Liu 16 302 0 09 Aug 2016
Gaussian Error Linear Units (GELUs) Dan Hendrycks Kevin Gimpel 46 4,865 0 27 Jun 2016
Convolutional Residual Memory Networks Joel Ruben Antony Moniz C. Pal 23 23 0 16 Jun 2016
Convolutional Neural Fabrics Shreyas Saxena Jakob Verbeek 21 225 0 08 Jun 2016
FractalNet: Ultra-Deep Neural Networks without Residuals Gustav Larsson Michael Maire Gregory Shakhnarovich 43 933 0 24 May 2016
Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks Tim Salimans Diederik P. Kingma ODL 54 1,924 0 25 Feb 2016
Revise Saturated Activation Functions Bing Xu Ruitong Huang Mu Li 9 72 0 18 Feb 2016