All you need is a good init

19 November 2015

Papers citing "All you need is a good init"

26 / 26 papers shown

Title
Sinusoidal Initialization, Time for a New Start Alberto Fernández-Hernández Jose I. Mestre Manuel F. Dolz Jose Duato Enrique S. Quintana-Ortí ODL AI4CE 138 0 0 19 May 2025
Shrinkage Initialization for Smooth Learning of Neural Networks Miao Cheng Feiyan Zhou Hongwei Zou Limin Wang AI4CE 61 0 0 12 Apr 2025
RoRA: Efficient Fine-Tuning of LLM with Reliability Optimization for Rank Adaptation Jun Liu Zhenglun Kong Peiyan Dong Changdi Yang Xuan Shen ... Wei Niu Wenbin Zhang Xue Lin Dong Huang Yanzhi Wang ALM 75 2 0 08 Jan 2025
Deep activity propagation via weight initialization in spiking neural networks Aurora Micheli Olaf Booij Jan van Gemert Nergis Tömen 61 0 0 01 Oct 2024
From Lazy to Rich: Exact Learning Dynamics in Deep Linear Networks Clémentine Dominé Nicolas Anguita A. Proca Lukas Braun D. Kunin P. Mediano Andrew M. Saxe 76 3 0 22 Sep 2024
Oja's plasticity rule overcomes several challenges of training neural networks under biological constraints Navid Shervani-Tabar Marzieh Alireza Mirhoseini Robert Rosenbaum AAML AI4CE 94 0 0 15 Aug 2024
All You Need is Beyond a Good Init: Exploring Better Solution for Training Extremely Deep Convolutional Neural Networks with Orthonormality and Modulation Di Xie Jiang Xiong Shiliang Pu 100 182 0 06 Mar 2017
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 1.6K 192,638 0 10 Dec 2015
Batch-normalized Maxout Network in Network Jia-Ren Chang Yonghao Chen OOD 75 108 0 09 Nov 2015
Very Deep Multilingual Convolutional Neural Networks for LVCSR Tom Sercu Christian Puhrsch Brian Kingsbury Yann LeCun 46 224 0 29 Sep 2015
Training Very Deep Networks R. Srivastava Klaus Greff Jürgen Schmidhuber 114 1,675 0 22 Jul 2015
Distilling the Knowledge in a Neural Network Geoffrey E. Hinton Oriol Vinyals J. Dean FedML 292 19,523 0 09 Mar 2015
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift Sergey Ioffe Christian Szegedy OOD 388 43,154 0 11 Feb 2015
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification Kaiming He Xinming Zhang Shaoqing Ren Jian Sun VLM 227 18,534 0 06 Feb 2015
Striving for Simplicity: The All Convolutional Net Jost Tobias Springenberg Alexey Dosovitskiy Thomas Brox Martin Riedmiller FAtt 202 4,653 0 21 Dec 2014
Random Walk Initialization for Training Very Deep Feedforward Networks David Sussillo L. F. Abbott 46 81 0 19 Dec 2014
FitNets: Hints for Thin Deep Nets Adriana Romero Nicolas Ballas Samira Ebrahimi Kahou Antoine Chassang C. Gatta Yoshua Bengio FedML 260 3,862 0 19 Dec 2014
Fractional Max-Pooling Benjamin Graham TPM 78 517 0 18 Dec 2014
Spatially-sparse convolutional neural networks Benjamin Graham 83 231 0 22 Sep 2014
Deeply-Supervised Nets Chen-Yu Lee Saining Xie Patrick W. Gallagher Zhengyou Zhang Zhuowen Tu 307 2,229 0 18 Sep 2014
Going Deeper with Convolutions Christian Szegedy Wei Liu Yangqing Jia P. Sermanet Scott E. Reed Dragomir Anguelov D. Erhan Vincent Vanhoucke Andrew Rabinovich 368 43,511 0 17 Sep 2014
Very Deep Convolutional Networks for Large-Scale Image Recognition Karen Simonyan Andrew Zisserman FAtt MDE 1.2K 99,991 0 04 Sep 2014
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 1.3K 39,383 0 01 Sep 2014
Caffe: Convolutional Architecture for Fast Feature Embedding Yangqing Jia Evan Shelhamer Jeff Donahue Sergey Karayev Jonathan Long Ross B. Girshick S. Guadarrama Trevor Darrell VLM BDL 3DV 231 14,703 0 20 Jun 2014
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks Andrew M. Saxe James L. McClelland Surya Ganguli ODL 143 1,830 0 20 Dec 2013
Maxout Networks Ian Goodfellow David Warde-Farley M. Berk Mirza Aaron Courville Yoshua Bengio OOD 206 2,176 0 18 Feb 2013