Exponentially vanishing sub-optimal local minima in multilayer neural networks

19 February 2017

Papers citing "Exponentially vanishing sub-optimal local minima in multilayer neural networks"

36 / 36 papers shown

Title
Loss Landscape of Shallow ReLU-like Neural Networks: Stationary Points, Saddle Escape, and Network Embedding Zhengqing Wu Berfin Simsek Francois Ged ODL 69 0 0 08 Feb 2024
Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approach Ryo Karakida S. Akaho S. Amari FedML 111 143 0 04 Jun 2018
When is a Convolutional Filter Easy To Learn? S. Du Jason D. Lee Yuandong Tian MLT 48 130 0 18 Sep 2017
Theoretical insights into the optimization landscape of over-parameterized shallow neural networks Mahdi Soltanolkotabi Adel Javanmard Jason D. Lee 116 417 0 16 Jul 2017
Recovery Guarantees for One-hidden-layer Neural Networks Kai Zhong Zhao Song Prateek Jain Peter L. Bartlett Inderjit S. Dhillon MLT 118 336 0 10 Jun 2017
Weight Sharing is Crucial to Succesful Optimization Shai Shalev-Shwartz Ohad Shamir Shaked Shammah 59 12 0 02 Jun 2017
Convergence Analysis of Two-layer Neural Networks with ReLU Activation Yuanzhi Li Yang Yuan MLT 101 650 0 28 May 2017
Train longer, generalize better: closing the generalization gap in large batch training of neural networks Elad Hoffer Itay Hubara Daniel Soudry ODL 142 799 0 24 May 2017
The Landscape of Deep Learning Algorithms Pan Zhou Jiashi Feng 44 24 0 19 May 2017
The loss surface of deep and wide neural networks Quynh N. Nguyen Matthias Hein ODL 89 284 0 26 Apr 2017
Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data Gintare Karolina Dziugaite Daniel M. Roy 82 808 0 31 Mar 2017
Depth Creates No Bad Local Minima Haihao Lu Kenji Kawaguchi ODL FAtt 57 121 0 27 Feb 2017
Globally Optimal Gradient Descent for a ConvNet with Gaussian Inputs Alon Brutzkus Amir Globerson MLT 110 313 0 26 Feb 2017
Local minima in training of neural networks G. Swirszcz Wojciech M. Czarnecki Razvan Pascanu ODL 51 73 0 19 Nov 2016
Towards a Mathematical Understanding of the Difficulty in Learning with Feedforward Neural Networks Hao Shen AAML 11 3 0 17 Nov 2016
Identity Matters in Deep Learning Moritz Hardt Tengyu Ma OOD 61 399 0 14 Nov 2016
Understanding deep learning requires rethinking generalization Chiyuan Zhang Samy Bengio Moritz Hardt Benjamin Recht Oriol Vinyals HAI 269 4,620 0 10 Nov 2016
Distribution-Specific Hardness of Learning Neural Networks Ohad Shamir 60 116 0 05 Sep 2016
Exponential expressivity in deep neural networks through transient chaos Ben Poole Subhaneil Lahiri M. Raghu Jascha Narain Sohl-Dickstein Surya Ganguli 83 587 0 16 Jun 2016
No bad local minima: Data independent training error guarantees for multilayer neural networks Daniel Soudry Y. Carmon 119 235 0 26 May 2016
Robust Large Margin Deep Neural Networks Jure Sokolić Raja Giryes Guillermo Sapiro M. Rodrigues 59 309 0 26 May 2016
Deep Learning without Poor Local Minima Kenji Kawaguchi ODL 162 922 0 23 May 2016
Gradient Descent Converges to Minimizers Jason D. Lee Max Simchowitz Michael I. Jordan Benjamin Recht 55 211 0 16 Feb 2016
Ensemble Robustness and Generalization of Stochastic Deep Learning Algorithms Tom Zahavy Bingyi Kang Alex Sivak Jiashi Feng Huan Xu Shie Mannor OOD AAML 48 12 0 07 Feb 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 1.4K 192,638 0 10 Dec 2015
On the Quality of the Initial Basin in Overspecified Neural Networks Itay Safran Ohad Shamir 53 127 0 13 Nov 2015
When Are Nonconvex Problems Not Scary? Ju Sun Qing Qu John N. Wright 57 166 0 21 Oct 2015
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification Kaiming He Xinming Zhang Shaoqing Ren Jian Sun VLM 200 18,534 0 06 Feb 2015
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 840 149,474 0 22 Dec 2014
Qualitatively characterizing neural network optimization problems Ian Goodfellow Oriol Vinyals Andrew M. Saxe ODL 83 519 0 19 Dec 2014
The Loss Surfaces of Multilayer Networks A. Choromańska Mikael Henaff Michaël Mathieu Gerard Ben Arous Yann LeCun ODL 230 1,191 0 30 Nov 2014
On the Computational Efficiency of Training Neural Networks Roi Livni Shai Shalev-Shwartz Ohad Shamir 76 479 0 05 Oct 2014
Identifying and attacking the saddle point problem in high-dimensional non-convex optimization Yann N. Dauphin Razvan Pascanu Çağlar Gülçehre Kyunghyun Cho Surya Ganguli Yoshua Bengio ODL 106 1,380 0 10 Jun 2014
One weird trick for parallelizing convolutional neural networks A. Krizhevsky GNN 81 1,297 0 23 Apr 2014
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks Andrew M. Saxe James L. McClelland Surya Ganguli ODL 122 1,830 0 20 Dec 2013
Identifiability of parameters in latent structure models with many observed variables E. Allman C. Matias J. Rhodes CML 118 532 0 29 Sep 2008