M-FAC: Efficient Matrix-Free Approximations of Second-Order Information

M-FAC: Efficient Matrix-Free Approximations of Second-Order Information

7 July 2021

Dan Alistarh

Papers citing "M-FAC: Efficient Matrix-Free Approximations of Second-Order Information"

11 / 11 papers shown

Title
Self-calibration for Language Model Quantization and Pruning Miles Williams G. Chrysostomou Nikolaos Aletras MQ 355 0 0 22 Oct 2024
4-bit Shampoo for Memory-Efficient Network Training Sike Wang Jia Li Pan Zhou Hua Huang MQ 78 8 0 28 May 2024
Always-Sparse Training by Growing Connections with Guided Stochastic Exploration Mike Heddes Narayan Srinivasa T. Givargis Alexandru Nicolau 156 0 0 12 Jan 2024
ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning Z. Yao A. Gholami Sheng Shen Mustafa Mustafa Kurt Keutzer Michael W. Mahoney ODL 71 280 0 01 Jun 2020
Soft Threshold Weight Reparameterization for Learnable Sparsity Aditya Kusupati Vivek Ramanujan Raghav Somani Mitchell Wortsman Prateek Jain Sham Kakade Ali Farhadi 90 243 0 08 Feb 2020
Limitations of the Empirical Fisher Approximation for Natural Gradient Descent Frederik Kunstner Lukas Balles Philipp Hennig 58 212 0 29 May 2019
EigenDamage: Structured Pruning in the Kronecker-Factored Eigenbasis Chaoqi Wang Roger C. Grosse Sanja Fidler Guodong Zhang 45 121 0 15 May 2019
Know What You Don't Know: Unanswerable Questions for SQuAD Pranav Rajpurkar Robin Jia Percy Liang RALM ELM 179 2,818 0 11 Jun 2018
To prune, or not to prune: exploring the efficacy of pruning for model compression Michael Zhu Suyog Gupta 123 1,262 0 05 Oct 2017
TensorFlow: A system for large-scale machine learning Martín Abadi P. Barham Jianmin Chen Zhiwen Chen Andy Davis ... Vijay Vasudevan Pete Warden Martin Wicke Yuan Yu Xiaoqiang Zhang GNN AI4CE 324 18,300 0 27 May 2016
Variational Dropout and the Local Reparameterization Trick Diederik P. Kingma Tim Salimans Max Welling BDL 149 1,500 0 08 Jun 2015