Gradient Descent on Neurons and its Link to Approximate Second-Order Optimization

28 January 2022

Papers citing "Gradient Descent on Neurons and its Link to Approximate Second-Order Optimization"

7 / 7 papers shown

Title
Position: Curvature Matrices Should Be Democratized via Linear Operators Felix Dangel Runa Eschenhagen Weronika Ormaniec Andres Fernandez Lukas Tatzel Agustinus Kristiadi 58 3 0 31 Jan 2025
Debiasing Mini-Batch Quadratics for Applications in Deep Learning Lukas Tatzel Bálint Mucsányi Osane Hackel Philipp Hennig 43 0 0 18 Oct 2024
An Improved Empirical Fisher Approximation for Natural Gradient Descent Xiaodong Wu Wenyi Yu Chao Zhang Philip Woodland 29 3 0 10 Jun 2024
AdaFisher: Adaptive Second Order Optimization via Fisher Information Damien Martins Gomes Yanlei Zhang Eugene Belilovsky Guy Wolf Mahdi S. Hosseini ODL 76 2 0 26 May 2024
Eva: A General Vectorized Approximation Framework for Second-order Optimization Lin Zhang S. Shi Bo-wen Li 28 1 0 04 Aug 2023
Random initialisations performing above chance and how to find them Frederik Benzing Simon Schug Robert Meier J. Oswald Yassir Akram Nicolas Zucchet Laurence Aitchison Angelika Steger ODL 35 24 0 15 Sep 2022
Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adam Mohammad Emtiyaz Khan Didrik Nielsen Voot Tangkaratt Wu Lin Y. Gal Akash Srivastava ODL 74 266 0 13 Jun 2018