GMP*: Well-Tuned Gradual Magnitude Pruning Can Outperform Most
BERT-Pruning Methods

GMP*: Well-Tuned Gradual Magnitude Pruning Can Outperform Most BERT-Pruning Methods

12 October 2022

Dan Alistarh

Papers citing "GMP*: Well-Tuned Gradual Magnitude Pruning Can Outperform Most BERT-Pruning Methods"

15 / 15 papers shown

Title
Pruning Large Language Models with Semi-Structural Adaptive Sparse Training Weiyu Huang Yuezhou Hu Guohao Jian Jun Zhu Jianfei Chen 35 5 0 30 Jul 2024
Beyond Perplexity: Multi-dimensional Safety Evaluation of LLM Compression Zhichao Xu Ashim Gupta Tao Li Oliver Bentham Vivek Srikumar 49 8 0 06 Jul 2024
SA-FedLora: Adaptive Parameter Allocation for Efficient Federated Learning with LoRA Tuning Yuning Yang Xiaohong Liu Tianrun Gao Xiaodong Xu Guangyu Wang 37 5 0 15 May 2024
Model Compression and Efficient Inference for Large Language Models: A Survey Wenxiao Wang Wei Chen Yicong Luo Yongliu Long Zhengkai Lin Liye Zhang Binbin Lin Deng Cai Xiaofei He MQ 41 47 0 15 Feb 2024
VENOM: A Vectorized N:M Format for Unleashing the Power of Sparse Tensor Cores Roberto L. Castro Andrei Ivanov Diego Andrade Tal Ben-Nun B. Fraguela Torsten Hoefler 19 15 0 03 Oct 2023
Scaling Laws for Sparsely-Connected Foundation Models Elias Frantar C. Riquelme N. Houlsby Dan Alistarh Utku Evci 30 35 0 15 Sep 2023
How Does Pruning Impact Long-Tailed Multi-Label Medical Image Classifiers? G. Holste Ziyu Jiang Ajay Jaiswal Maria Hanna Shlomo Minkowitz ... Ying Ding Ronald M. Summers George Shih Yifan Peng Zhangyang Wang 26 1 0 17 Aug 2023
Accurate Neural Network Pruning Requires Rethinking Sparse Optimization Denis Kuznedelev Eldar Kurtic Eugenia Iofinova Elias Frantar Alexandra Peste Dan Alistarh VLM 35 11 0 03 Aug 2023
Towards Automated Circuit Discovery for Mechanistic Interpretability Arthur Conmy Augustine N. Mavor-Parker Aengus Lynch Stefan Heimersheim Adrià Garriga-Alonso 23 280 0 28 Apr 2023
Bias in Pruned Vision Models: In-Depth Analysis and Countermeasures Eugenia Iofinova Alexandra Peste Dan Alistarh 28 9 0 25 Apr 2023
oBERTa: Improving Sparse Transfer Learning via improved initialization, distillation, and pruning regimes Daniel Fernando Campos Alexandre Marques Mark Kurtz Chengxiang Zhai VLM AAML 13 2 0 30 Mar 2023
ZipLM: Inference-Aware Structured Pruning of Language Models Eldar Kurtic Elias Frantar Dan Alistarh MQ 25 24 0 07 Feb 2023
SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot Elias Frantar Dan Alistarh VLM 33 627 0 02 Jan 2023
Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks Torsten Hoefler Dan Alistarh Tal Ben-Nun Nikoli Dryden Alexandra Peste MQ 141 684 0 31 Jan 2021
The Lottery Ticket Hypothesis for Pre-trained BERT Networks Tianlong Chen Jonathan Frankle Shiyu Chang Sijia Liu Yang Zhang Zhangyang Wang Michael Carbin 153 345 0 23 Jul 2020