The Hessian perspective into the Nature of Convolutional Neural Networks

16 May 2023

Papers citing "The Hessian perspective into the Nature of Convolutional Neural Networks"

21 / 21 papers shown

Title
Accelerating Neural Network Training Along Sharp and Flat Directions Daniyar Zakarin Sidak Pal Singh ODL 68 0 0 17 May 2025
Theoretical characterisation of the Gauss-Newton conditioning in Neural Networks Jim Zhao Sidak Pal Singh Aurelien Lucchi AI4CE 116 0 0 04 Nov 2024
What Does It Mean to Be a Transformer? Insights from a Theoretical Hessian Analysis Weronika Ormaniec Felix Dangel Sidak Pal Singh 113 7 0 14 Oct 2024
Analytic Insights into Structure and Rank of Neural Network Hessian Maps Sidak Pal Singh Gregor Bachmann Thomas Hofmann FAtt 70 37 0 30 Jun 2021
MLP-Mixer: An all-MLP Architecture for Vision Ilya O. Tolstikhin N. Houlsby Alexander Kolesnikov Lucas Beyer Xiaohua Zhai ... Andreas Steiner Daniel Keysers Jakob Uszkoreit Mario Lucic Alexey Dosovitskiy 423 2,682 0 04 May 2021
Dissecting Hessian: Understanding Common Structure of Hessian in Neural Networks Yikai Wu Xingyu Zhu Chenwei Wu Annie Wang Rong Ge 103 45 0 08 Oct 2020
Theory of Deep Convolutional Neural Networks II: Spherical Analysis Zhiying Fang Han Feng Shuo Huang Ding-Xuan Zhou 94 37 0 28 Jul 2020
PyHessian: Neural Networks Through the Lens of the Hessian Z. Yao A. Gholami Kurt Keutzer Michael W. Mahoney ODL 63 303 0 16 Dec 2019
Fantastic Generalization Measures and Where to Find Them Yiding Jiang Behnam Neyshabur H. Mobahi Dilip Krishnan Samy Bengio AI4CE 139 610 0 04 Dec 2019
Information-Theoretic Local Minima Characterization and Regularization Zhiwei Jia Hao Su 64 19 0 19 Nov 2019
Limitations of the Empirical Fisher Approximation for Natural Gradient Descent Frederik Kunstner Lukas Balles Philipp Hennig 81 218 0 29 May 2019
1D Convolutional Neural Networks and Applications: A Survey S. Kiranyaz Onur Avcı Osama Abdeljaber T. Ince Moncef Gabbouj D. Inman 3DV 85 1,930 0 09 May 2019
An Investigation into Neural Net Optimization via Hessian Eigenvalue Density Behrooz Ghorbani Shankar Krishnan Ying Xiao ODL 78 326 0 29 Jan 2019
Gradient Descent Happens in a Tiny Subspace Guy Gur-Ari Daniel A. Roberts Ethan Dyer 100 233 0 12 Dec 2018
The Singular Values of Convolutional Layers Hanie Sedghi Vineet Gupta Philip M. Long FAtt 86 203 0 26 May 2018
Overcoming catastrophic forgetting in neural networks J. Kirkpatrick Razvan Pascanu Neil C. Rabinowitz J. Veness Guillaume Desjardins ... A. Grabska-Barwinska Demis Hassabis Claudia Clopath D. Kumaran R. Hadsell CLL 372 7,547 0 02 Dec 2016
Eigenvalues of the Hessian in Deep Learning: Singularity and Beyond Levent Sagun Léon Bottou Yann LeCun UQCV 91 236 0 22 Nov 2016
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 429 2,945 0 15 Sep 2016
Optimizing Neural Networks with Kronecker-factored Approximate Curvature James Martens Roger C. Grosse ODL 104 1,023 0 19 Mar 2015
Identifying and attacking the saddle point problem in high-dimensional non-convex optimization Yann N. Dauphin Razvan Pascanu Çağlar Gülçehre Kyunghyun Cho Surya Ganguli Yoshua Bengio ODL 129 1,389 0 10 Jun 2014
Invariant Scattering Convolution Networks Joan Bruna S. Mallat 129 1,279 0 05 Mar 2012