Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approach

4 June 2018

Papers citing "Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approach"

50 / 96 papers shown

Title
More Optimal Fractional-Order Stochastic Gradient Descent for Non-Convex Optimization Problems Mohammad Partohaghighi Roummel Marcia YangQuan Chen 12 0 0 05 May 2025
Enhancing Multi-task Learning Capability of Medical Generalist Foundation Model via Image-centric Multi-annotation Data Xun Zhu Fanbin Mo Zheng Zhang J. Wang Yiming Shi Ming Wu Chuang Zhang Miao Li Ji Wu 32 0 0 14 Apr 2025
Distributed Log-driven Anomaly Detection System based on Evolving Decision Making Zhuoran Tan Qiyuan Wang Christos Anagnostopoulos S. P. Parambath Jeremy Singer Sam Temple 43 0 0 03 Apr 2025
Effective Dimension Aware Fractional-Order Stochastic Gradient Descent for Convex Optimization Problems Mohammad Partohaghighi Roummel Marcia YangQuan Chen 37 0 0 17 Mar 2025
NoT: Federated Unlearning via Weight Negation Yasser H. Khalil Leo Maxime Brunswic Soufiane Lamghari Xu Li Mahdi Beitollahi Xi Chen MU 48 2 0 07 Mar 2025
Robust and Efficient Deep Hedging via Linearized Objective Neural Network Lei Zhao Lin Cai 69 0 0 25 Feb 2025
Fishing For Cheap And Efficient Pruners At Initialization Ivo Gollini Navarrete Nicolas Mauricio Cuadrado Jose Renato Restom Martin Takáč Samuel Horvath 44 0 0 17 Feb 2025
Evidence on the Regularisation Properties of Maximum-Entropy Reinforcement Learning Rémy Hosseinkhan Boucher Onofrio Semeraro L. Mathelin 74 0 0 28 Jan 2025
Fisher Information-based Efficient Curriculum Federated Learning with Large Language Models Ji Liu Jiaxiang Ren Ruoming Jin Zijie Zhang Yang Zhou P. Valduriez Dejing Dou FedML 31 1 0 30 Sep 2024
ClassiFIM: An Unsupervised Method To Detect Phase Transitions Victor Kasatkin E. Mozgunov Nicholas Ezzell Utkarsh Mishra Itay Hen Daniel Lidar 24 2 0 06 Aug 2024
Towards the Spectral bias Alleviation by Normalizations in Coordinate Networks Zhicheng Cai Hao Zhu Qiu Shen Xinran Wang Xun Cao 41 0 0 25 Jul 2024
Task2Box: Box Embeddings for Modeling Asymmetric Task Relationships Rangel Daroya Aaron Sun Subhransu Maji 27 0 0 25 Mar 2024
A Differential Geometric View and Explainability of GNN on Evolving Graphs Yazheng Liu Xi Zhang Sihong Xie 19 3 0 11 Mar 2024
Tradeoffs of Diagonal Fisher Information Matrix Estimators Alexander Soen Ke Sun 14 1 0 08 Feb 2024
On the Parameterization of Second-Order Optimization Effective Towards the Infinite Width Satoki Ishikawa Ryo Karakida 24 2 0 19 Dec 2023
Temperature Balancing, Layer-wise Weight Analysis, and Neural Network Training Yefan Zhou Tianyu Pang Keqin Liu Charles H. Martin Michael W. Mahoney Yaoqing Yang 34 7 0 01 Dec 2023
SiGeo: Sub-One-Shot NAS via Information Theory and Geometry of Loss Landscape Hua Zheng Kuang-Hung Liu Igor Fedorov Xin Zhang Wen-Yen Chen Wei Wen 36 1 0 22 Nov 2023
Symmetry Induces Structure and Constraint of Learning Liu Ziyin 26 10 0 29 Sep 2023
Deep Unsupervised Learning Using Spike-Timing-Dependent Plasticity Sen Lu Abhronil Sengupta BDL 18 7 0 08 Jul 2023
AutoTransfer: AutoML with Knowledge Transfer -- An Application to Graph Neural Networks Kaidi Cao Jiaxuan You Jiaju Liu J. Leskovec 59 4 0 14 Mar 2023
Bayesian inference with finitely wide neural networks Chi-Ken Lu BDL 37 0 0 06 Mar 2023
Phase diagram of early training dynamics in deep neural networks: effect of the learning rate, depth, and width Dayal Singh Kalra M. Barkeshli 15 9 0 23 Feb 2023
Structural Neural Additive Models: Enhanced Interpretable Machine Learning Mattias Luber Anton Thielmann Benjamin Säfken 27 7 0 18 Feb 2023
$Generalization Ability of Wide Neural Networks on $\mathbb{R}$$ Generalization Ability of Wide Neural Networks on $\mathbb{R}$ Jianfa Lai Manyun Xu Rui Chen Qi-Rong Lin 26 21 0 12 Feb 2023
Efficient Activation Function Optimization through Surrogate Modeling G. Bingham Risto Miikkulainen 16 2 0 13 Jan 2023
Maximal Initial Learning Rates in Deep ReLU Networks Gaurav M. Iyer Boris Hanin David Rolnick 21 9 0 14 Dec 2022
Exploring Temporal Information Dynamics in Spiking Neural Networks Youngeun Kim Yuhang Li Hyoungseob Park Yeshwanth Venkatesha Anna Hambitzer Priyadarshini Panda 19 32 0 26 Nov 2022
Characterizing the Spectrum of the NTK via a Power Series Expansion Michael Murray Hui Jin Benjamin Bowman Guido Montúfar 32 11 0 15 Nov 2022
FIT: A Metric for Model Sensitivity Ben Zandonati Adrian Alan Pol M. Pierini Olya Sirkin Tal Kopetz MQ 24 8 0 16 Oct 2022
Foundation Transformers Hongyu Wang Shuming Ma Shaohan Huang Li Dong Wenhui Wang ... Barun Patra Zhun Liu Vishrav Chaudhary Xia Song Furu Wei AI4CE 27 27 0 12 Oct 2022
Component-Wise Natural Gradient Descent -- An Efficient Neural Network Optimization Tran van Sang Mhd Irvan R. Yamaguchi Toshiyuki Nakata 13 1 0 11 Oct 2022
Hyperbolic VAE via Latent Gaussian Distributions Seunghyuk Cho Juyong Lee Dongwoo Kim DRL 39 5 0 30 Sep 2022
Visualizing high-dimensional loss landscapes with Hessian directions Lucas Böttcher Gregory R. Wheeler 19 12 0 28 Aug 2022
Policy Gradients using Variational Quantum Circuits André Sequeira L. Santos Luis Soares Barbosa 39 11 0 20 Mar 2022
Classical versus Quantum: comparing Tensor Network-based Quantum Circuits on LHC data Jack Y. Araz M. Spannowsky 28 14 0 21 Feb 2022
Deep Learning in Random Neural Fields: Numerical Experiments via Neural Tangent Kernel Kaito Watanabe Kotaro Sakamoto Ryo Karakida Sho Sonoda S. Amari OOD 19 1 0 10 Feb 2022
A Local Geometric Interpretation of Feature Extraction in Deep Feedforward Neural Networks Md Kamran Chowdhury Shisher Tasmeen Zaman Ornee Yin Sun MILM FAtt 11 2 0 09 Feb 2022
Neural Tangent Kernel Beyond the Infinite-Width Limit: Effects of Depth and Initialization Mariia Seleznova Gitta Kutyniok 179 16 0 01 Feb 2022
A generalization gap estimation for overparameterized models via the Langevin functional variance Akifumi Okuno Keisuke Yano 38 1 0 07 Dec 2021
Approximate Spectral Decomposition of Fisher Information Matrix for Simple ReLU Networks Yoshinari Takeishi Masazumi Iida J. Takeuchi 9 4 0 30 Nov 2021
Does the Data Induce Capacity Control in Deep Learning? Rubing Yang J. Mao Pratik Chaudhari 25 15 0 27 Oct 2021
Feature Learning and Signal Propagation in Deep Neural Networks Yizhang Lou Chris Mingard Yoonsoo Nam Soufiane Hayou MDE 6 17 0 22 Oct 2021
Analytic Study of Families of Spurious Minima in Two-Layer ReLU Neural Networks: A Tale of Symmetry II Yossi Arjevani M. Field 28 18 0 21 Jul 2021
On the Variance of the Fisher Information for Deep Learning Alexander Soen Ke Sun FedML FAtt 6 15 0 09 Jul 2021
Analytic Insights into Structure and Rank of Neural Network Hessian Maps Sidak Pal Singh Gregor Bachmann Thomas Hofmann FAtt 9 32 0 30 Jun 2021
TENGraD: Time-Efficient Natural Gradient Descent with Exact Fisher-Block Inversion Saeed Soori Bugra Can Baourun Mu Mert Gurbuzbalaban M. Dehnavi 24 10 0 07 Jun 2021
Lower Bounds on the Generalization Error of Nonlinear Learning Models Inbar Seroussi Ofer Zeitouni 13 5 0 26 Mar 2021
Asymptotic Freeness of Layerwise Jacobians Caused by Invariance of Multilayer Perceptron: The Haar Orthogonal Case B. Collins Tomohiro Hayase 22 7 0 24 Mar 2021
Non-asymptotic approximations of neural networks by Gaussian processes Ronen Eldan Dan Mikulincer T. Schramm 33 24 0 17 Feb 2021
Analyzing Finite Neural Networks: Can We Trust Neural Tangent Kernel Theory? Mariia Seleznova Gitta Kutyniok AAML 16 29 0 08 Dec 2020