Revisiting Natural Gradient for Deep Networks

16 January 2013

Papers citing "Revisiting Natural Gradient for Deep Networks"

29 / 229 papers shown

Title
Reducing the Model Order of Deep Neural Networks Using Information Theory Ming Tu Visar Berisha Yu Cao Jae-sun Seo 11 23 0 16 May 2016
Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) Djork-Arné Clevert Thomas Unterthiner Sepp Hochreiter 19 5,481 0 23 Nov 2015
Data-Dependent Path Normalization in Neural Networks Behnam Neyshabur Ryota Tomioka Ruslan Salakhutdinov Nathan Srebro 21 22 0 20 Nov 2015
Symmetry-invariant optimization in deep networks Vijay Badrinarayanan Bamdev Mishra R. Cipolla ODL 19 31 0 05 Nov 2015
adaQN: An Adaptive Quasi-Newton Algorithm for Training RNNs N. Keskar A. Berahas ODL 9 34 0 04 Nov 2015
Understanding symmetries in deep networks Vijay Badrinarayanan Bamdev Mishra R. Cipolla 221 42 0 03 Nov 2015
Faster Stochastic Variational Inference using Proximal-Gradient Methods with General Divergence Functions Mohammad Emtiyaz Khan Reza Babanezhad Wu Lin Mark W. Schmidt Masashi Sugiyama 28 49 0 31 Oct 2015
Hessian-free Optimization for Learning Deep Multidimensional Recurrent Neural Networks Minhyung Cho C. Dhir Jaehyung Lee 11 8 0 11 Sep 2015
Fast Second-Order Stochastic Backpropagation for Variational Inference Kai Fan Ziteng Wang J. Beck James T. Kwok Katherine A. Heller ODL BDL DRL 23 45 0 09 Sep 2015
Training Conditional Random Fields with Natural Gradient Descent Yuan Cao BDL 14 0 0 10 Aug 2015
Natural Neural Networks Guillaume Desjardins Karen Simonyan Razvan Pascanu Koray Kavukcuoglu 17 176 0 01 Jul 2015
Faster SGD Using Sketched Conditioning Alon Gonen Shai Shalev-Shwartz 18 16 0 08 Jun 2015
Optimizing Neural Networks with Kronecker-factored Approximate Curvature James Martens Roger C. Grosse ODL 33 984 0 19 Mar 2015
Trust Region Policy Optimization John Schulman Sergey Levine Philipp Moritz Michael I. Jordan Pieter Abbeel 32 6,664 0 19 Feb 2015
Equilibrated adaptive learning rates for non-convex optimization Yann N. Dauphin H. D. Vries Yoshua Bengio ODL 22 375 0 15 Feb 2015
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 22 148,823 0 22 Dec 2014
New insights and perspectives on the natural gradient method James Martens ODL 17 602 0 03 Dec 2014
Identifying and attacking the saddle point problem in high-dimensional non-convex optimization Yann N. Dauphin Razvan Pascanu Çağlar Gülçehre Kyunghyun Cho Surya Ganguli Yoshua Bengio ODL 29 1,376 0 10 Jun 2014
Iterative Neural Autoregressive Distribution Estimator (NADE-k) T. Raiko L. Yao Kyunghyun Cho Yoshua Bengio 24 42 0 05 Jun 2014
On the saddle point problem for non-convex optimization Razvan Pascanu Yann N. Dauphin Surya Ganguli Yoshua Bengio ODL 29 105 0 19 May 2014
On the Number of Linear Regions of Deep Neural Networks Guido Montúfar Razvan Pascanu Kyunghyun Cho Yoshua Bengio 29 1,239 0 08 Feb 2014
How to Construct Deep Recurrent Neural Networks Razvan Pascanu Çağlar Gülçehre Kyunghyun Cho Yoshua Bengio 48 1,008 0 20 Dec 2013
Learned-Norm Pooling for Deep Feedforward and Recurrent Neural Networks Çağlar Gülçehre Kyunghyun Cho Razvan Pascanu Yoshua Bengio 40 170 0 07 Nov 2013
SKYNET: an efficient and robust neural network training tool for machine learning in astronomy P. Graff F. Feroz Michael P. Hobson A. Lasenby 33 95 0 03 Sep 2013
Riemannian metrics for neural networks II: recurrent networks and learning symbolic data sequences Yann Ollivier 31 17 0 03 Jun 2013
Deep Learning of Representations: Looking Forward Yoshua Bengio 40 680 0 02 May 2013
Riemannian metrics for neural networks I: feedforward networks Yann Ollivier 31 102 0 04 Mar 2013
Training Neural Networks with Stochastic Hessian-Free Optimization Ryan Kiros BDL 49 48 0 16 Jan 2013
Representation Learning: A Review and New Perspectives Yoshua Bengio Aaron Courville Pascal Vincent OOD SSL 29 12,331 0 24 Jun 2012