Natural Neural Networks

1 July 2015

Papers citing "Natural Neural Networks"

49 / 49 papers shown

Title
Budgeted Online Continual Learning by Adaptive Layer Freezing and Frequency-based Sampling Minhyuk Seo Hyunseo Koh Jonghyun Choi 39 1 0 19 Oct 2024
Efficient Deep Learning with Decorrelated Backpropagation Sander Dalm Joshua Offergeld Nasir Ahmad Marcel van Gerven 54 4 0 03 May 2024
VNE: An Effective Method for Improving Deep Representation by Manipulating Eigenvalue Distribution Jaeill Kim Suhyun Kang Duhun Hwang Jungwook Shin Wonjong Rhee DRL 13 21 0 04 Apr 2023
The Geometry of Neural Nets' Parameter Spaces Under Reparametrization Agustinus Kristiadi Felix Dangel Philipp Hennig 32 11 0 14 Feb 2023
Random initialisations performing above chance and how to find them Frederik Benzing Simon Schug Robert Meier J. Oswald Yassir Akram Nicolas Zucchet Laurence Aitchison Angelika Steger ODL 35 24 0 15 Sep 2022
What do CNNs Learn in the First Layer and Why? A Linear Systems Perspective Rhea Chowers Yair Weiss 33 2 0 06 Jun 2022
Feedback Gradient Descent: Efficient and Stable Optimization with Orthogonality for DNNs Fanchen Bu D. Chang 28 6 0 12 May 2022
Approximate Nearest Neighbor Search under Neural Similarity Metric for Large-Scale Recommendation Rihan Chen Bin Liu Ziru Xu Yao Wang Qi Li ... Q. hua Junliang Jiang Yunlong Xu Hongbo Deng Bo Zheng 40 21 0 14 Feb 2022
Gradient Descent on Neurons and its Link to Approximate Second-Order Optimization Frederik Benzing ODL 43 23 0 28 Jan 2022
Low-Pass Filtering SGD for Recovering Flat Optima in the Deep Learning Optimization Landscape Devansh Bisla Jing Wang A. Choromańska 25 34 0 20 Jan 2022
Learnable Adaptive Cosine Estimator (LACE) for Image Classification Joshua Peeples Connor H. McCurley Sarah Walker Dylan Stewart Alina Zare 24 3 0 11 Oct 2021
Powerpropagation: A sparsity inducing weight reparameterisation Jonathan Richard Schwarz Siddhant M. Jayakumar Razvan Pascanu P. Latham Yee Whye Teh 92 54 0 01 Oct 2021
Batch Normalization Preconditioning for Neural Network Training Susanna Lange Kyle E. Helfrich Qiang Ye 27 9 0 02 Aug 2021
TENGraD: Time-Efficient Natural Gradient Descent with Exact Fisher-Block Inversion Saeed Soori Bugra Can Baourun Mu Mert Gurbuzbalaban M. Dehnavi 24 10 0 07 Jun 2021
Stochastic Whitening Batch Normalization Shengdong Zhang E. Nezhadarya H. Fashandi Jiayi Liu Darin Graham Mohak Shah 21 10 0 03 Jun 2021
Shapley Explanation Networks Rui Wang Xiaoqian Wang David I. Inouye TDI FAtt 27 44 0 06 Apr 2021
Information Geometry and Classical Cramér-Rao Type Inequalities Kumar Vijay Mishra M. I. M. Ashok Kumar 16 1 0 02 Apr 2021
EigenGame: PCA as a Nash Equilibrium I. Gemp Brian McWilliams Claire Vernade T. Graepel 24 46 0 01 Oct 2020
Group Whitening: Balancing Learning Efficiency and Representational Capacity Lei Huang Yi Zhou Li Liu Fan Zhu Ling Shao 28 21 0 28 Sep 2020
Transform Quantization for CNN (Convolutional Neural Network) Compression Sean I. Young Wang Zhe David S. Taubman B. Girod MQ 29 69 0 02 Sep 2020
Whitening and second order optimization both make information in the dataset unusable during training, and can reduce or prevent generalization Neha S. Wadia Daniel Duckworth S. Schoenholz Ethan Dyer Jascha Narain Sohl-Dickstein 27 13 0 17 Aug 2020
Practical Quasi-Newton Methods for Training Deep Neural Networks D. Goldfarb Yi Ren Achraf Bahamou ODL 10 104 0 16 Jun 2020
Incremental Object Detection via Meta-Learning K. J. Joseph Jathushan Rajasegaran Salman Khan Fahad Shahbaz Khan V. Balasubramanian ObjD CLL VLM 177 98 0 17 Mar 2020
Out-of-Distribution Generalization via Risk Extrapolation (REx) David M. Krueger Ethan Caballero J. Jacobsen Amy Zhang Jonathan Binas Dinghuai Zhang Rémi Le Priol Aaron Courville OOD 215 901 0 02 Mar 2020
Concept Whitening for Interpretable Image Recognition Zhi Chen Yijie Bei Cynthia Rudin FAtt 31 314 0 05 Feb 2020
Orthogonal Wasserstein GANs J. Müller Reinhard Klein Michael Weinmann 48 9 0 29 Nov 2019
Switchable Normalization for Learning-to-Normalize Deep Representation Ping Luo Ruimao Zhang Jiamin Ren Zhanglin Peng Jingyu Li 30 73 0 22 Jul 2019
Backpropagation-Friendly Eigendecomposition Wei Wang Zheng Dang Yinlin Hu Pascal Fua Mathieu Salzmann 17 49 0 21 Jun 2019
Natural Option Critic Saket Tiwari Philip S. Thomas 17 22 0 04 Dec 2018
Deep Frank-Wolfe For Neural Network Optimization Leonard Berrada Andrew Zisserman M. P. Kumar ODL 18 40 0 19 Nov 2018
Mode Normalization Lucas Deecke Iain Murray Hakan Bilen OOD 29 33 0 12 Oct 2018
Combining Natural Gradient with Hessian Free Methods for Sequence Training Adnan Haider P. Woodland ODL 20 4 0 03 Oct 2018
Fast Approximate Natural Gradient Descent in a Kronecker-factored Eigenbasis Thomas George César Laurent Xavier Bouthillier Nicolas Ballas Pascal Vincent ODL 29 150 0 11 Jun 2018
Meta-Learning with Hessian-Free Approach in Deep Neural Nets Training Boyu Chen Wenlian Lu Ernest Fokoue 21 1 0 22 May 2018
Decorrelated Batch Normalization Lei Huang Dawei Yang B. Lang Jia Deng 16 190 0 23 Apr 2018
OLÉ: Orthogonal Low-rank Embedding, A Plug and Play Geometric Loss for Deep Learning José Lezama Qiang Qiu Pablo Musé Guillermo Sapiro 35 78 0 05 Dec 2017
Self-Normalizing Neural Networks Günter Klambauer Thomas Unterthiner Andreas Mayr Sepp Hochreiter 82 2,485 0 08 Jun 2017
Sharp Minima Can Generalize For Deep Nets Laurent Dinh Razvan Pascanu Samy Bengio Yoshua Bengio ODL 46 757 0 15 Mar 2017
Gaussian Error Linear Units (GELUs) Dan Hendrycks Kevin Gimpel 90 4,873 0 27 Jun 2016
Relative Natural Gradient for Learning Large Complex Models Ke Sun Frank Nielsen 29 5 0 20 Jun 2016
Parametric Exponential Linear Unit for Deep Convolutional Neural Networks Ludovic Trottier Philippe Giguère B. Chaib-draa 41 199 0 30 May 2016
Normalization Propagation: A Parametric Technique for Removing Internal Covariate Shift in Deep Networks Devansh Arpit Yingbo Zhou Bhargava U. Kota V. Govindaraju 21 126 0 04 Mar 2016
Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks Tim Salimans Diederik P. Kingma ODL 99 1,924 0 25 Feb 2016
Learning values across many orders of magnitude H. V. Hasselt A. Guez Matteo Hessel Volodymyr Mnih David Silver 17 169 0 24 Feb 2016
A Kronecker-factored approximate Fisher matrix for convolution layers Roger C. Grosse James Martens ODL 40 258 0 03 Feb 2016
Scalable Gradient-Based Tuning of Continuous Regularization Hyperparameters Jelena Luketina Mathias Berglund Klaus Greff T. Raiko 29 173 0 20 Nov 2015
Symmetry-invariant optimization in deep networks Vijay Badrinarayanan Bamdev Mishra R. Cipolla ODL 19 31 0 05 Nov 2015
Batch Normalized Recurrent Neural Networks César Laurent Gabriel Pereyra Philemon Brakel Wenjie Qu Yoshua Bengio 35 213 0 05 Oct 2015
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift Sergey Ioffe Christian Szegedy OOD 88 43,015 0 11 Feb 2015