Practical Quasi-Newton Methods for Training Deep Neural Networks

16 June 2020

Papers citing "Practical Quasi-Newton Methods for Training Deep Neural Networks"

20 / 20 papers shown

Title
Striving for Simplicity: Simple Yet Effective Prior-Aware Pseudo-Labeling for Semi-Supervised Ultrasound Image Segmentation Yaxiong Chen Yujie Wang Zixuan Zheng Jingliang Hu Yilei Shi Shengwu Xiong Xiao Xiang Zhu Lichao Mou 54 0 0 18 Mar 2025
Learning rheological parameters of non-Newtonian fluids from velocimetry data Alexandros Kontogiannis Richard Hodgkinson E. L. Manchester 16 0 0 05 Aug 2024
An Improved Empirical Fisher Approximation for Natural Gradient Descent Xiaodong Wu Wenyi Yu Chao Zhang Philip Woodland 29 3 0 10 Jun 2024
Q-Newton: Hybrid Quantum-Classical Scheduling for Accelerating Neural Network Training with Newton's Gradient Descent Pingzhi Li Junyu Liu Hanrui Wang Tianlong Chen 94 1 0 30 Apr 2024
Eva: A General Vectorized Approximation Framework for Second-order Optimization Lin Zhang S. Shi Bo-wen Li 28 1 0 04 Aug 2023
KrADagrad: Kronecker Approximation-Domination Gradient Preconditioned Stochastic Optimization Jonathan Mei Alexander Moreno Luke Walters ODL 29 1 0 30 May 2023
Layer-wise Adaptive Step-Sizes for Stochastic First-Order Methods for Deep Learning Achraf Bahamou D. Goldfarb ODL 36 0 0 23 May 2023
ASDL: A Unified Interface for Gradient Preconditioning in PyTorch Kazuki Osawa Satoki Ishikawa Rio Yokota Shigang Li Torsten Hoefler ODL 46 14 0 08 May 2023
ISAAC Newton: Input-based Approximate Curvature for Newton's Method Felix Petersen Tobias Sutter Christian Borgelt Dongsung Huh Hilde Kuehne Yuekai Sun Oliver Deussen ODL 36 5 0 01 May 2023
FOSI: Hybrid First and Second Order Optimization Hadar Sivan Moshe Gabel Assaf Schuster ODL 34 2 0 16 Feb 2023
An Efficient Nonlinear Acceleration method that Exploits Symmetry of the Hessian Huan He Shifan Zhao Z. Tang Joyce C. Ho Y. Saad Yuanzhe Xi 32 3 0 22 Oct 2022
Rethinking Exponential Averaging of the Fisher C. Puiu 23 1 0 10 Apr 2022
Gradient Descent on Neurons and its Link to Approximate Second-Order Optimization Frederik Benzing ODL 43 23 0 28 Jan 2022
Large-Scale Deep Learning Optimizations: A Comprehensive Survey Xiaoxin He Fuzhao Xue Xiaozhe Ren Yang You 30 14 0 01 Nov 2021
Accelerating Distributed K-FAC with Smart Parallelism of Computing and Communication Tasks S. Shi Lin Zhang Bo-wen Li 40 9 0 14 Jul 2021
KOALA: A Kalman Optimization Algorithm with Loss Adaptivity A. Davtyan Sepehr Sameni L. Cerkezi Givi Meishvili Adam Bielski Paolo Favaro ODL 55 2 0 07 Jul 2021
NG+ : A Multi-Step Matrix-Product Natural Gradient Method for Deep Learning Minghan Yang Dong Xu Qiwen Cui Zaiwen Wen Pengxiang Xu 18 4 0 14 Jun 2021
TENGraD: Time-Efficient Natural Gradient Descent with Exact Fisher-Block Inversion Saeed Soori Bugra Can Baourun Mu Mert Gurbuzbalaban M. Dehnavi 24 10 0 07 Jun 2021
A Trace-restricted Kronecker-Factored Approximation to Natural Gradient Kai-Xin Gao Xiaolei Liu Zheng-Hai Huang Min Wang Zidong Wang Dachuan Xu F. Yu 24 11 0 21 Nov 2020
Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers Robin M. Schmidt Frank Schneider Philipp Hennig ODL 40 162 0 03 Jul 2020