Negative eigenvalues of the Hessian in deep neural networks

6 February 2019

Nicolas Le Roux

Papers citing "Negative eigenvalues of the Hessian in deep neural networks"

28 / 28 papers shown

Title
Towards Quantifying the Hessian Structure of Neural Networks Zhaorui Dong Yushun Zhang Zhi-Quan Luo Jianfeng Yao Ruoyu Sun 31 0 0 05 May 2025
Parameter Symmetry Breaking and Restoration Determines the Hierarchical Learning in AI Systems Liu Ziyin Yizhou Xu T. Poggio Isaac Chuang 52 4 0 07 Feb 2025
Nesterov acceleration in benignly non-convex landscapes Kanan Gupta Stephan Wojtowytsch 42 2 0 10 Oct 2024
Visualizing, Rethinking, and Mining the Loss Landscape of Deep Neural Networks Xin-Chun Li Lan Li De-Chuan Zhan 41 2 0 21 May 2024
A qualitative difference between gradient flows of convex functions in finite- and infinite-dimensional Hilbert spaces Jonathan W. Siegel Stephan Wojtowytsch 21 3 0 26 Oct 2023
Symmetry Induces Structure and Constraint of Learning Liu Ziyin 34 10 0 29 Sep 2023
Unveiling the Hessian's Connection to the Decision Boundary Mahalakshmi Sabanayagam Freya Behrens Urte Adomaityte Anna Dawid 30 5 0 12 Jun 2023
Revisiting the Fragility of Influence Functions Jacob R. Epifano Ravichandran Ramachandran A. Masino Ghulam Rasool TDI 27 14 0 22 Mar 2023
Escaping Saddle Points for Effective Generalization on Class-Imbalanced Data Harsh Rangwani Sumukh K Aithal Mayank Mishra R. Venkatesh Babu 31 29 0 28 Dec 2022
Improving Robust Generalization by Direct PAC-Bayesian Bound Minimization Zifa Wang Nan Ding Tomer Levinboim Xi Chen Radu Soricut AAML 37 5 0 22 Nov 2022
Visualizing high-dimensional loss landscapes with Hessian directions Lucas Böttcher Gregory R. Wheeler 37 13 0 28 Aug 2022
Curvature-informed multi-task learning for graph networks Alexander New M. Pekala Nam Q. Le Janna Domenico C. Piatko Christopher D. Stiles 25 4 0 02 Aug 2022
Diffusion Curvature for Estimating Local Curvature in High Dimensional Data Dhananjay Bhaskar Kincaid MacDonald O. Fasina Dawson Thomas Bastian Alexander Rieck Ian M. Adelstein Smita Krishnaswamy DiffM 25 7 0 08 Jun 2022
Complexity from Adaptive-Symmetries Breaking: Global Minima in the Statistical Mechanics of Deep Neural Networks Shaun Li AI4CE 46 0 0 03 Jan 2022
SGD with a Constant Large Learning Rate Can Converge to Local Maxima Liu Ziyin Botao Li James B. Simon Masakuni Ueda 29 8 0 25 Jul 2021
Combating Mode Collapse in GAN training: An Empirical Analysis using Hessian Eigenvalues Ricard Durall Avraam Chatzimichailidis P. Labus J. Keuper GAN 30 58 0 17 Dec 2020
Traces of Class/Cross-Class Structure Pervade Deep Learning Spectra Vardan Papyan 14 76 0 27 Aug 2020
The Break-Even Point on Optimization Trajectories of Deep Neural Networks Stanislaw Jastrzebski Maciej Szymczak Stanislav Fort Devansh Arpit Jacek Tabor Kyunghyun Cho Krzysztof J. Geras 50 155 0 21 Feb 2020
DDPNOpt: Differential Dynamic Programming Neural Optimizer Guan-Horng Liu T. Chen Evangelos A. Theodorou 27 7 0 20 Feb 2020
Low Rank Saddle Free Newton: A Scalable Method for Stochastic Nonconvex Optimization Thomas O'Leary-Roseberry Nick Alger Omar Ghattas ODL 37 9 0 07 Feb 2020
Epistemic Uncertainty Quantification in Deep Learning Classification by the Delta Method G. K. Nilsen A. Munthe-Kaas H. Skaug M. Brun UQCV 8 0 0 02 Dec 2019
SGD momentum optimizer with step estimation by online parabola model J. Duda ODL 21 22 0 16 Jul 2019
A Closer Look at the Optimization Landscapes of Generative Adversarial Networks Hugo Berard Gauthier Gidel Amjad Almahairi Pascal Vincent Simon Lacoste-Julien GAN 20 64 0 11 Jun 2019
A Geometric Modeling of Occam's Razor in Deep Learning Ke Sun Frank Nielsen 16 5 0 27 May 2019
Adaptive norms for deep learning with regularized Newton methods Jonas Köhler Leonard Adolphs Aurelien Lucchi ODL 9 11 0 22 May 2019
Improving SGD convergence by online linear regression of gradients in multiple statistically relevant directions J. Duda ODL 12 1 0 31 Jan 2019
Interpreting Adversarial Robustness: A View from Decision Surface in Input Space Fuxun Yu Chenchen Liu Yanzhi Wang Liang Zhao Xiang Chen AAML OOD 36 27 0 29 Sep 2018
The Loss Surfaces of Multilayer Networks A. Choromańska Mikael Henaff Michaël Mathieu Gerard Ben Arous Yann LeCun ODL 186 1,186 0 30 Nov 2014