DenoiseRotator: Enhance Pruning Robustness for LLMs via Importance Concentration

29 May 2025

Papers citing "DenoiseRotator: Enhance Pruning Robustness for LLMs via Importance Concentration"

25 / 25 papers shown

Title
SpinQuant: LLM quantization with learned rotations Zechun Liu Changsheng Zhao Igor Fedorov Bilge Soran Dhruv Choudhary Raghuraman Krishnamoorthi Vikas Chandra Yuandong Tian Tijmen Blankevoort MQ 167 105 0 21 Feb 2025
OstQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fitting Xing Hu Yuan Cheng Dawei Yang Zukang Xu Zhihang Yuan Jiangyong Yu Chen Xu Zhe Jiang Sifan Zhou MQ 69 9 0 23 Jan 2025
MaskLLM: Learnable Semi-Structured Sparsity for Large Language Models Gongfan Fang Hongxu Yin Saurav Muralidharan Greg Heinrich Jeff Pool Jan Kautz Pavlo Molchanov Xinchao Wang 57 5 0 26 Sep 2024
VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Models Yifei Liu Jicheng Wen Yang Wang Shengyu Ye Li Lyna Zhang Ting Cao Cheng Li Mao Yang MQ 174 12 0 25 Sep 2024
QTIP: Quantization with Trellises and Incoherence Processing Albert Tseng Qingyao Sun David Hou Christopher De Sa MQ 63 12 0 17 Jun 2024
Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for Large Language Models Peijie Dong Lujun Li Zhenheng Tang Xiang Liu Xinglin Pan Qiang-qiang Wang Xiaowen Chu 88 29 0 05 Jun 2024
QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs Saleh Ashkboos Amirkeivan Mohtashami Maximilian L. Croci Bo Li Martin Jaggi Dan Alistarh Torsten Hoefler James Hensman MQ 68 159 0 30 Mar 2024
QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks Albert Tseng Jerry Chee Qingyao Sun Volodymyr Kuleshov Christopher De Sa MQ 155 110 0 06 Feb 2024
SliceGPT: Compress Large Language Models by Deleting Rows and Columns Saleh Ashkboos Maximilian L. Croci Marcelo Gennari do Nascimento Torsten Hoefler James Hensman VLM 158 163 0 26 Jan 2024
A Simple and Effective Pruning Approach for Large Language Models Mingjie Sun Zhuang Liu Anna Bair J. Zico Kolter 97 397 0 20 Jun 2023
LLM-Pruner: On the Structural Pruning of Large Language Models Xinyin Ma Gongfan Fang Xinchao Wang 78 395 0 19 May 2023
SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot Elias Frantar Dan Alistarh VLM 75 677 0 02 Jan 2023
Geoopt: Riemannian Optimization in PyTorch Max Kochurov R. Karimov Sergei Kozlukov 45 118 0 06 May 2020
Efficient Riemannian Optimization on the Stiefel Manifold via the Cayley Transform Jun Li Fuxin Li S. Todorovic 33 104 0 04 Feb 2020
PyTorch: An Imperative Style, High-Performance Deep Learning Library Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury ... Sasank Chilamkurthy Benoit Steiner Lu Fang Junjie Bai Soumith Chintala ODL 248 42,038 0 03 Dec 2019
PIQA: Reasoning about Physical Commonsense in Natural Language Yonatan Bisk Rowan Zellers Ronan Le Bras Jianfeng Gao Yejin Choi OOD LRM 79 1,724 0 26 Nov 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 260 19,824 0 23 Oct 2019
HellaSwag: Can a Machine Really Finish Your Sentence? Rowan Zellers Ari Holtzman Yonatan Bisk Ali Farhadi Yejin Choi 67 2,373 0 19 May 2019
Riemannian Adaptive Optimization Methods Gary Bécigneul O. Ganea ODL 85 254 0 01 Oct 2018
Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge Peter Clark Isaac Cowhey Oren Etzioni Tushar Khot Ashish Sabharwal Carissa Schoenick Oyvind Tafjord ELM RALM LRM 69 2,474 0 14 Mar 2018
Learning Sparse Neural Networks through $L_0$ Regularization Christos Louizos Max Welling Diederik P. Kingma 236 1,136 0 04 Dec 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 437 129,831 0 12 Jun 2017
Pointer Sentinel Mixture Models Stephen Merity Caiming Xiong James Bradbury R. Socher RALM 161 2,814 0 26 Sep 2016
Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding Song Han Huizi Mao W. Dally 3DGS 200 8,793 0 01 Oct 2015
Distilling the Knowledge in a Neural Network Geoffrey E. Hinton Oriol Vinyals J. Dean FedML 236 19,523 0 09 Mar 2015