The loss surface of deep linear networks viewed through the algebraic geometry lens

17 October 2018

Papers citing "The loss surface of deep linear networks viewed through the algebraic geometry lens"

50 / 56 papers shown

Title
Understanding Mode Connectivity via Parameter Space Symmetry B. Zhao Nima Dehmamy Robin Walters Rose Yu 211 7 0 29 May 2025
Minnorm training: an algorithm for training over-parameterized deep neural networks Yamini Bansal Madhu S. Advani David D. Cox Andrew M. Saxe ODL 38 18 0 03 Jun 2018
The Global Optimization Geometry of Shallow Linear Neural Networks Zhihui Zhu Daniel Soudry Yonina C. Eldar M. Wakin ODL 54 36 0 13 May 2018
Using Machine Learning to Improve Cylindrical Algebraic Decomposition Zongyan Huang Matthew England D. Wilson J. Davenport Lawrence Charles Paulson 47 24 0 26 Apr 2018
The Loss Surface of XOR Artificial Neural Networks D. Mehta Xiaojun Zhao Edgar A. Bernal D. Wales 125 19 0 06 Apr 2018
Energy-entropy competition and the effectiveness of stochastic gradient descent in machine learning Yao Zhang Andrew M. Saxe Madhu S. Advani A. Lee 52 60 0 05 Mar 2018
Small nonlinearities in activation functions create bad local minima in neural networks Chulhee Yun S. Sra Ali Jadbabaie ODL 71 95 0 10 Feb 2018
BPGrad: Towards Global Optimality in Deep Learning via Branch and Pruning Ziming Zhang Yuanwei Wu Guanghui Wang ODL 55 28 0 19 Nov 2017
Three Factors Influencing Minima in SGD Stanislaw Jastrzebski Zachary Kenton Devansh Arpit Nicolas Ballas Asja Fischer Yoshua Bengio Amos Storkey 76 463 0 13 Nov 2017
Critical Points of Neural Networks: Analytical Forms and Landscape Properties Yi Zhou Yingbin Liang 55 54 0 30 Oct 2017
First-order Methods Almost Always Avoid Saddle Points Jason D. Lee Ioannis Panageas Georgios Piliouras Max Simchowitz Michael I. Jordan Benjamin Recht ODL 124 83 0 20 Oct 2017
Characterization of Gradient Dominance and Regularity Conditions for Neural Networks Yi Zhou Yingbin Liang 62 33 0 18 Oct 2017
Generalization in Deep Learning Kenji Kawaguchi L. Kaelbling Yoshua Bengio ODL 86 459 0 16 Oct 2017
Porcupine Neural Networks: (Almost) All Local Optima are Global Soheil Feizi Hamid Javadi Jesse M. Zhang David Tse 68 36 0 05 Oct 2017
How regularization affects the critical points in linear networks Amirhossein Taghvaei Jin-Won Kim P. Mehta 63 13 0 27 Sep 2017
Global optimality conditions for deep neural networks Chulhee Yun S. Sra Ali Jadbabaie 136 118 0 08 Jul 2017
Towards Understanding Generalization of Deep Learning: Perspective of Loss Landscapes Lei Wu Zhanxing Zhu E. Weinan ODL 62 221 0 30 Jun 2017
Empirical Analysis of the Hessian of Over-Parametrized Neural Networks Levent Sagun Utku Evci V. U. Güney Yann N. Dauphin Léon Bottou 54 418 0 14 Jun 2017
Are Saddles Good Enough for Deep Learning? Adepu Ravi Sankar V. Balasubramanian 61 5 0 07 Jun 2017
Deep Complex Networks C. Trabelsi O. Bilaniuk Ying Zhang Dmitriy Serdyuk Sandeep Subramanian J. F. Santos Soroush Mehri Negar Rostamzadeh Yoshua Bengio C. Pal 180 834 0 27 May 2017
The loss surface of deep and wide neural networks Quynh N. Nguyen Matthias Hein ODL 154 284 0 26 Apr 2017
Sharp Minima Can Generalize For Deep Nets Laurent Dinh Razvan Pascanu Samy Bengio Yoshua Bengio ODL 112 772 0 15 Mar 2017
Depth Creates No Bad Local Minima Haihao Lu Kenji Kawaguchi ODL FAtt 71 121 0 27 Feb 2017
Exponentially vanishing sub-optimal local minima in multilayer neural networks Daniel Soudry Elad Hoffer 144 97 0 19 Feb 2017
Skip Connections Eliminate Singularities Emin Orhan Xaq Pitkow 56 25 0 31 Jan 2017
An empirical analysis of the optimization of deep network loss surfaces Daniel Jiwoong Im Michael Tao K. Branson ODL 55 63 0 13 Dec 2016
Eigenvalues of the Hessian in Deep Learning: Singularity and Beyond Levent Sagun Léon Bottou Yann LeCun UQCV 84 236 0 22 Nov 2016
Local minima in training of neural networks G. Swirszcz Wojciech M. Czarnecki Razvan Pascanu ODL 61 73 0 19 Nov 2016
Identity Matters in Deep Learning Moritz Hardt Tengyu Ma OOD 81 398 0 14 Nov 2016
Entropy-SGD: Biasing Gradient Descent Into Wide Valleys Pratik Chaudhari A. Choromańska Stefano Soatto Yann LeCun Carlo Baldassi C. Borgs J. Chayes Levent Sagun R. Zecchina ODL 96 773 0 06 Nov 2016
Topology and Geometry of Half-Rectified Network Optimization C. Freeman Joan Bruna 192 235 0 04 Nov 2016
Understanding Deep Neural Networks with Rectified Linear Units R. Arora A. Basu Poorya Mianjy Anirbit Mukherjee PINN 143 641 0 04 Nov 2016
Full-Capacity Unitary Recurrent Neural Networks Scott Wisdom Thomas Powers J. Hershey Jonathan Le Roux L. Atlas 52 293 0 31 Oct 2016
Homotopy Analysis for Tensor PCA Anima Anandkumar Yuan-bei Deng Rong Ge H. Mobahi 56 43 0 28 Oct 2016
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 421 2,937 0 15 Sep 2016
No bad local minima: Data independent training error guarantees for multilayer neural networks Daniel Soudry Y. Carmon 183 235 0 26 May 2016
Deep Learning without Poor Local Minima Kenji Kawaguchi ODL 219 923 0 23 May 2016
Fixed Points of Belief Propagation -- An Analysis via Polynomial Homotopy Continuation Christian Knoll Franz Pernkopf D. Mehta Tianran Chen 48 17 0 20 May 2016
Unreasonable Effectiveness of Learning Neural Networks: From Accessible States and Robust Ensembles to Basic Algorithmic Schemes Carlo Baldassi C. Borgs J. Chayes Alessandro Ingrosso Carlo Lucibello Luca Saglietti R. Zecchina 60 168 0 20 May 2016
On Complex Valued Convolutional Neural Networks Nitzan Guberman CVBM 70 133 0 29 Feb 2016
Efficient approaches for escaping higher order saddle points in non-convex optimization Anima Anandkumar Rong Ge 28 143 0 18 Feb 2016
Learning may need only a few bits of synaptic precision Carlo Baldassi Federica Gerace Carlo Lucibello Luca Saglietti R. Zecchina 46 28 0 12 Feb 2016
Associative Long Short-Term Memory Ivo Danihelka Greg Wayne Benigno Uria Nal Kalchbrenner Alex Graves 63 178 0 09 Feb 2016
On the energy landscape of deep networks Pratik Chaudhari Stefano Soatto ODL 82 27 0 20 Nov 2015
Unitary Evolution Recurrent Neural Networks Martín Arjovsky Amar Shah Yoshua Bengio ODL 75 770 0 20 Nov 2015
Local entropy as a measure for sampling solutions in Constraint Satisfaction Problems Carlo Baldassi Alessandro Ingrosso Carlo Lucibello Luca Saglietti R. Zecchina 41 58 0 18 Nov 2015
Global Optimality in Tensor Factorization, Deep Learning, and Beyond B. Haeffele René Vidal 178 150 0 24 Jun 2015
Path-SGD: Path-Normalized Optimization in Deep Neural Networks Behnam Neyshabur Ruslan Salakhutdinov Nathan Srebro ODL 79 307 0 08 Jun 2015
Escaping From Saddle Points --- Online Stochastic Gradient for Tensor Decomposition Rong Ge Furong Huang Chi Jin Yang Yuan 135 1,058 0 06 Mar 2015
Explorations on high dimensional landscapes Levent Sagun V. U. Güney Gerard Ben Arous Yann LeCun 52 65 0 20 Dec 2014