Eigenvalues of the Hessian in Deep Learning: Singularity and Beyond

Eigenvalues of the Hessian in Deep Learning: Singularity and Beyond

22 November 2016

Papers citing "Eigenvalues of the Hessian in Deep Learning: Singularity and Beyond"

18 / 68 papers shown

Title
Geometry of learning neural quantum states Chae-Yeun Park M. Kastoryano 29 60 0 24 Oct 2019
Asymptotics of Wide Networks from Feynman Diagrams Ethan Dyer Guy Gur-Ari 29 113 0 25 Sep 2019
Hessian based analysis of SGD for Deep Nets: Dynamics and Generalization Xinyan Li Qilong Gu Yingxue Zhou Tiancong Chen A. Banerjee ODL 42 51 0 24 Jul 2019
Weight-space symmetry in deep networks gives rise to permutation saddles, connected by equal-loss valleys across the loss landscape Johanni Brea Berfin Simsek Bernd Illing W. Gerstner 23 55 0 05 Jul 2019
First Exit Time Analysis of Stochastic Gradient Descent Under Heavy-Tailed Gradient Noise T. H. Nguyen Umut Simsekli Mert Gurbuzbalaban G. Richard 12 59 0 21 Jun 2019
Negative eigenvalues of the Hessian in deep neural networks Guillaume Alain Nicolas Le Roux Pierre-Antoine Manzagol 21 42 0 06 Feb 2019
An Investigation into Neural Net Optimization via Hessian Eigenvalue Density Behrooz Ghorbani Shankar Krishnan Ying Xiao ODL 18 317 0 29 Jan 2019
Measurements of Three-Level Hierarchical Structure in the Outliers in the Spectrum of Deepnet Hessians Vardan Papyan 30 87 0 24 Jan 2019
Breaking Reversibility Accelerates Langevin Dynamics for Global Non-Convex Optimization Xuefeng Gao Mert Gurbuzbalaban Lingjiong Zhu 22 30 0 19 Dec 2018
Gradient Descent Happens in a Tiny Subspace Guy Gur-Ari Daniel A. Roberts Ethan Dyer 30 229 0 12 Dec 2018
The Full Spectrum of Deepnet Hessians at Scale: Dynamics with SGD Training and Sample Size Vardan Papyan 9 31 0 16 Nov 2018
Universal discriminative quantum neural networks Hongxiang Chen Leonard Wossnig Simone Severini Hartmut Neven Masoud Mohseni 21 80 0 22 May 2018
Local Saddle Point Optimization: A Curvature Exploitation Approach Leonard Adolphs Hadi Daneshmand Aurelien Lucchi Thomas Hofmann 37 107 0 15 May 2018
Comparing Dynamics: Deep Neural Networks versus Glassy Systems Marco Baity-Jesi Levent Sagun Mario Geiger S. Spigler Gerard Ben Arous C. Cammarota Yann LeCun M. Wyart Giulio Biroli AI4CE 42 113 0 19 Mar 2018
High Dimensional Spaces, Deep Learning and Adversarial Examples S. Dube 37 29 0 02 Jan 2018
Neumann Optimizer: A Practical Optimization Algorithm for Deep Neural Networks Shankar Krishnan Ying Xiao Rif A. Saurous ODL 22 19 0 08 Dec 2017
The loss surface of deep and wide neural networks Quynh N. Nguyen Matthias Hein ODL 51 283 0 26 Apr 2017
Sharp Minima Can Generalize For Deep Nets Laurent Dinh Razvan Pascanu Samy Bengio Yoshua Bengio ODL 46 758 0 15 Mar 2017