Emergent properties of the local geometry of neural loss landscapes

14 October 2019

Papers citing "Emergent properties of the local geometry of neural loss landscapes"

43 / 43 papers shown

Title
Towards Quantifying the Hessian Structure of Neural Networks Zhaorui Dong Yushun Zhang Zhi-Quan Luo Jianfeng Yao Ruoyu Sun 33 0 0 05 May 2025
The effects of Hessian eigenvalue spectral density type on the applicability of Hessian analysis to generalization capability assessment of neural networks Nikita Gabdullin 23 0 0 24 Apr 2025
Improving Generalization with Flat Hilbert Bayesian Inference Tuan Truong Quyen Tran Quan Pham-Ngoc Nhat Ho Dinh Q. Phung Trung Le 31 0 0 05 Oct 2024
Unraveling the Hessian: A Key to Smooth Convergence in Loss Function Landscapes Nikita Kiselev Andrey Grabovoy 59 1 0 18 Sep 2024
Data Shapley in One Training Run Jiachen T. Wang Prateek Mittal Dawn Song Ruoxi Jia TDI 52 7 0 16 Jun 2024
Agnostic Sharpness-Aware Minimization Van-Anh Nguyen Quyen Tran Tuan Truong Thanh-Toan Do Dinh Q. Phung Trung Le 55 0 0 11 Jun 2024
Visualizing, Rethinking, and Mining the Loss Landscape of Deep Neural Networks Xin-Chun Li Lan Li De-Chuan Zhan 46 2 0 21 May 2024
Diversity-Aware Agnostic Ensemble of Sharpness Minimizers Anh-Vu Bui Vy Vo Tung Pham Dinh Q. Phung Trung Le FedML UQCV 39 1 0 19 Mar 2024
Deconstructing the Goldilocks Zone of Neural Network Initialization Artem Vysogorets Anna Dawid Julia Kempe 46 1 0 05 Feb 2024
Merging by Matching Models in Task Parameter Subspaces Derek Tam Mohit Bansal Colin Raffel MoMe 25 10 0 07 Dec 2023
Directions of Curvature as an Explanation for Loss of Plasticity Alex Lewandowski Haruto Tanaka Dale Schuurmans Marlos C. Machado 32 5 0 30 Nov 2023
Robust Contrastive Learning With Theory Guarantee Ngoc N. Tran Lam C. Tran Hoang Phan Anh-Vu Bui Tung Pham Toan M. Tran Dinh Q. Phung Trung Le SSL NoLa 34 0 0 16 Nov 2023
Outliers with Opposing Signals Have an Outsized Effect on Neural Network Optimization Elan Rosenfeld Andrej Risteski 30 10 0 07 Nov 2023
Low-Dimensional Gradient Helps Out-of-Distribution Detection Yingwen Wu Tao Li Xinwen Cheng Jie Yang Xiaolin Huang OODD 62 3 0 26 Oct 2023
Unveiling the Hessian's Connection to the Decision Boundary Mahalakshmi Sabanayagam Freya Behrens Urte Adomaityte Anna Dawid 38 5 0 12 Jun 2023
Catapults in SGD: spikes in the training loss and their impact on generalization through feature learning Libin Zhu Chaoyue Liu Adityanarayanan Radhakrishnan M. Belkin 43 14 0 07 Jun 2023
Optimal Transport Model Distributional Robustness Van-Anh Nguyen Trung Le Anh Tuan Bui Thanh-Toan Do Dinh Q. Phung OOD 36 3 0 07 Jun 2023
Phase diagram of early training dynamics in deep neural networks: effect of the learning rate, depth, and width Dayal Singh Kalra M. Barkeshli 25 9 0 23 Feb 2023
Flat Seeking Bayesian Neural Networks Van-Anh Nguyen L. Vuong Hoang Phan Thanh-Toan Do Dinh Q. Phung Trung Le BDL 44 8 0 06 Feb 2023
Improving Multi-task Learning via Seeking Task-based Flat Regions Hoang Phan Lam C. Tran Ngoc N. Tran Nhat Ho Dinh Q. Phung Trung Le 38 11 0 24 Nov 2022
On the Power-Law Hessian Spectrums in Deep Learning Zeke Xie Qian-Yuan Tang Yunfeng Cai Mingming Sun P. Li ODL 42 9 0 31 Jan 2022
Complexity from Adaptive-Symmetries Breaking: Global Minima in the Statistical Mechanics of Deep Neural Networks Shaun Li AI4CE 48 0 0 03 Jan 2022
Does the Data Induce Capacity Control in Deep Learning? Rubing Yang Jialin Mao Pratik Chaudhari 35 15 0 27 Oct 2021
On the Impact of Stable Ranks in Deep Nets B. Georgiev L. Franken Mayukh Mukherjee Georgios Arvanitidis 23 3 0 05 Oct 2021
Taxonomizing local versus global structure in neural network loss landscapes Yaoqing Yang Liam Hodgkinson Ryan Theisen Joe Zou Joseph E. Gonzalez Kannan Ramchandran Michael W. Mahoney 45 37 0 23 Jul 2021
Deep Learning on a Data Diet: Finding Important Examples Early in Training Mansheej Paul Surya Ganguli Gintare Karolina Dziugaite 19 438 0 15 Jul 2021
How many degrees of freedom do we need to train deep networks: a loss landscape perspective Brett W. Larsen Stanislav Fort Nico Becker Surya Ganguli UQCV 13 27 0 13 Jul 2021
Analyzing Monotonic Linear Interpolation in Neural Network Loss Landscapes James Lucas Juhan Bae Michael Ruogu Zhang Stanislav Fort R. Zemel Roger C. Grosse MoMe 172 28 0 22 Apr 2021
Hessian Eigenspectra of More Realistic Nonlinear Models Zhenyu Liao Michael W. Mahoney 27 29 0 02 Mar 2021
Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability Jeremy M. Cohen Simran Kaur Yuanzhi Li J. Zico Kolter Ameet Talwalkar ODL 43 253 0 26 Feb 2021
Consensus Control for Decentralized Deep Learning Lingjing Kong Tao R. Lin Anastasia Koloskova Martin Jaggi Sebastian U. Stich 19 76 0 09 Feb 2021
Quasi-Newton's method in the class gradient defined high-curvature subspace Mark Tuddenham Adam Prugel-Bennett Jonathan Hare ODL 31 7 0 28 Nov 2020
Chaos and Complexity from Quantum Neural Network: A study with Diffusion Metric in Machine Learning S. Choudhury Ankan Dutta Debisree Ray 22 21 0 16 Nov 2020
Improving Neural Network Training in Low Dimensional Random Bases Frithjof Gressmann Zach Eaton-Rosen Carlo Luschi 30 28 0 09 Nov 2020
Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the Neural Tangent Kernel Stanislav Fort Gintare Karolina Dziugaite Mansheej Paul Sepideh Kharaghani Daniel M. Roy Surya Ganguli 22 183 0 28 Oct 2020
Linear Mode Connectivity in Multitask and Continual Learning Seyed Iman Mirzadeh Mehrdad Farajtabar Dilan Görür Razvan Pascanu H. Ghasemzadeh CLL 37 139 0 09 Oct 2020
Dissecting Hessian: Understanding Common Structure of Hessian in Neural Networks Yikai Wu Xingyu Zhu Chenwei Wu Annie Wang Rong Ge 30 42 0 08 Oct 2020
Traces of Class/Cross-Class Structure Pervade Deep Learning Spectra Vardan Papyan 16 76 0 27 Aug 2020
Data-driven effective model shows a liquid-like deep learning Wenxuan Zou Haiping Huang 29 2 0 16 Jul 2020
Understanding the Role of Training Regimes in Continual Learning Seyed Iman Mirzadeh Mehrdad Farajtabar Razvan Pascanu H. Ghasemzadeh CLL 21 219 0 12 Jun 2020
The Break-Even Point on Optimization Trajectories of Deep Neural Networks Stanislaw Jastrzebski Maciej Szymczak Stanislav Fort Devansh Arpit Jacek Tabor Kyunghyun Cho Krzysztof J. Geras 55 155 0 21 Feb 2020
Stiffness: A New Perspective on Generalization in Neural Networks Stanislav Fort Pawel Krzysztof Nowak Stanislaw Jastrzebski S. Narayanan 27 94 0 28 Jan 2019
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 312 2,896 0 15 Sep 2016