KAISA: An Adaptive Second-Order Optimizer Framework for Deep Neural
Networks

KAISA: An Adaptive Second-Order Optimizer Framework for Deep Neural Networks

4 July 2021

Shivaram Venkataraman

Papers citing "KAISA: An Adaptive Second-Order Optimizer Framework for Deep Neural Networks"

10 / 10 papers shown

Title
Accelerating Deep Neural Network Training via Distributed Hybrid Order Optimization Shunxian Gu Chaoqun You Bangbang Ren Lailong Luo Junxu Xia Deke Guo 44 0 0 02 May 2025
SLoPe: Double-Pruned Sparse Plus Lazy Low-Rank Adapter Pretraining of LLMs Mohammad Mozaffari Amir Yazdanbakhsh Zhao Zhang M. Dehnavi 93 5 0 28 Jan 2025
Influence Functions for Scalable Data Attribution in Diffusion Models Bruno Mlodozeniec Runa Eschenhagen Juhan Bae Alexander Immer David Krueger Richard E. Turner TDI DiffM 75 4 0 17 Oct 2024
Eva: A General Vectorized Approximation Framework for Second-order Optimization Lin Zhang Shaoshuai Shi Bo Li 33 1 0 04 Aug 2023
On Efficient Training of Large-Scale Deep Learning Models: A Literature Review Li Shen Yan Sun Zhiyuan Yu Liang Ding Xinmei Tian Dacheng Tao VLM 35 41 0 07 Apr 2023
FOSI: Hybrid First and Second Order Optimization Hadar Sivan Moshe Gabel Assaf Schuster ODL 37 2 0 16 Feb 2023
PipeFisher: Efficient Training of Large Language Models Using Pipelining and Fisher Information Matrices Kazuki Osawa Shigang Li Torsten Hoefler AI4CE 35 24 0 25 Nov 2022
Efficient Quantized Sparse Matrix Operations on Tensor Cores Shigang Li Kazuki Osawa Torsten Hoefler 82 31 0 14 Sep 2022
Scalable K-FAC Training for Deep Neural Networks with Distributed Preconditioning Lin Zhang Shaoshuai Shi Wei Wang Bo Li 38 10 0 30 Jun 2022
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu M. Schuster Zhehuai Chen Quoc V. Le Mohammad Norouzi ... Alex Rudnick Oriol Vinyals G. Corrado Macduff Hughes J. Dean AIMat 718 6,750 0 26 Sep 2016