PyHessian: Neural Networks Through the Lens of the Hessian

PyHessian: Neural Networks Through the Lens of the Hessian

16 December 2019

Michael W. Mahoney

Papers citing "PyHessian: Neural Networks Through the Lens of the Hessian"

18 / 18 papers shown

Title
Unlocking the Potential of Unlabeled Data in Semi-Supervised Domain Generalization Dongkwan Lee Kyomin Hwang Nojun Kwak 130 0 0 18 Mar 2025
A Hessian-informed hyperparameter optimization for differential learning rate Shiyun Xu Zhiqi Bu Yiliang Zhang Ian Barnett 75 1 0 12 Jan 2025
Meta Curvature-Aware Minimization for Domain Generalization Zhaoyu Chen Yiwen Ye Feilong Tang Yongsheng Pan Yong-quan Xia BDL 352 1 0 16 Dec 2024
Sketched Adaptive Federated Deep Learning: A Sharp Convergence Analysis Zhijie Chen Qiaobo Li A. Banerjee FedML 69 0 0 11 Nov 2024
Theoretical characterisation of the Gauss-Newton conditioning in Neural Networks Jim Zhao Sidak Pal Singh Aurelien Lucchi AI4CE 88 0 0 04 Nov 2024
What Does It Mean to Be a Transformer? Insights from a Theoretical Hessian Analysis Weronika Ormaniec Felix Dangel Sidak Pal Singh 99 7 0 14 Oct 2024
Fast Training of Sinusoidal Neural Fields via Scaling Initialization Taesun Yeom Sangyoon Lee Jaeho Lee 83 3 0 07 Oct 2024
Loss Gradient Gaussian Width based Generalization and Optimization Guarantees A. Banerjee Qiaobo Li Yingxue Zhou 99 0 0 11 Jun 2024
Achieving Dimension-Free Communication in Federated Learning via Zeroth-Order Optimization Zhe Li Bicheng Ying Zidong Liu Chaosheng Dong Haibo Yang FedML 112 3 0 24 May 2024
Q-Newton: Hybrid Quantum-Classical Scheduling for Accelerating Neural Network Training with Newton's Gradient Descent Pingzhi Li Junyu Liu Hanrui Wang Tianlong Chen 137 2 0 30 Apr 2024
ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning Z. Yao A. Gholami Sheng Shen Mustafa Mustafa Kurt Keutzer Michael W. Mahoney ODL 91 281 0 01 Jun 2020
PowerNorm: Rethinking Batch Normalization in Transformers Sheng Shen Z. Yao A. Gholami Michael W. Mahoney Kurt Keutzer BDL 73 16 0 17 Mar 2020
HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks Zhen Dong Z. Yao Yaohui Cai Daiyaan Arfeen A. Gholami Michael W. Mahoney Kurt Keutzer MQ 77 279 0 10 Nov 2019
Inefficiency of K-FAC for Large Batch Size Training Linjian Ma Gabe Montague Jiayu Ye Z. Yao A. Gholami Kurt Keutzer Michael W. Mahoney 49 24 0 14 Mar 2019
Eigenvalues of the Hessian in Deep Learning: Singularity and Beyond Levent Sagun Léon Bottou Yann LeCun UQCV 81 236 0 22 Nov 2016
Sub-sampled Newton Methods with Non-uniform Sampling Peng Xu Jiyan Yang Farbod Roosta-Khorasani Christopher Ré Michael W. Mahoney 60 115 0 02 Jul 2016
Second-Order Stochastic Optimization for Machine Learning in Linear Time Naman Agarwal Brian Bullins Elad Hazan ODL 46 102 0 12 Feb 2016
Sub-Sampled Newton Methods I: Globally Convergent Algorithms Farbod Roosta-Khorasani Michael W. Mahoney 61 89 0 18 Jan 2016

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.