Extreme Compression of Large Language Models via Additive Quantization

Extreme Compression of Large Language Models via Additive Quantization

11 January 2024

Vage Egiazarian

Andrei Panferov

Denis Kuznedelev

Dan Alistarh

Papers citing "Extreme Compression of Large Language Models via Additive Quantization"

18 / 18 papers shown

Title
Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantization Yamato Arai Yuma Ichikawa MQ 66 0 0 13 Apr 2025
SpinQuant: LLM quantization with learned rotations Zechun Liu Changsheng Zhao Igor Fedorov Bilge Soran Dhruv Choudhary Raghuraman Krishnamoorthi Vikas Chandra Yuandong Tian Tijmen Blankevoort MQ 180 108 0 21 Feb 2025
ParetoQ: Scaling Laws in Extremely Low-bit LLM Quantization Zechun Liu Changsheng Zhao Hanxian Huang Sijia Chen Jing Zhang ... Yuandong Tian Bilge Soran Raghuraman Krishnamoorthi Tijmen Blankevoort Vikas Chandra MQ 118 8 0 04 Feb 2025
Fast Matrix Multiplications for Lookup Table-Quantized LLMs Han Guo William Brandon Radostin Cholakov Jonathan Ragan-Kelley Eric P. Xing Yoon Kim MQ 119 15 0 20 Jan 2025
GQSA: Group Quantization and Sparsity for Accelerating Large Language Model Inference Chao Zeng Songwei Liu Shu Yang Fangmin Chen Xing Mei Lean Fu MQ 63 0 0 23 Dec 2024
Mixture Compressor for Mixture-of-Experts LLMs Gains More Wei Huang Yue Liao Jianhui Liu Ruifei He Haoru Tan Shiming Zhang Hongsheng Li Si Liu Xiaojuan Qi MoE 58 4 0 08 Oct 2024
OATS: Outlier-Aware Pruning Through Sparse and Low Rank Decomposition Stephen Zhang Vardan Papyan VLM 117 3 0 20 Sep 2024
SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuning Yuze Zhao Jintao Huang Jinghan Hu Xingjun Wang Yunlin Mao ... Zhikai Wu Baole Ai Ang Wang Wenmeng Zhou Yingda Chen 69 38 0 10 Aug 2024
OAC: Output-adaptive Calibration for Accurate Post-training Quantization Ali Edalati Alireza Ghaffari M. Asgharian Lu Hou Boxing Chen Vahid Partovi Nia V. Nia MQ 105 0 0 23 May 2024
SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Models Wei Huang Haotong Qin Yangdong Liu Yawei Li Qinshuo Liu Xianglong Liu Luca Benini Michele Magno Shiming Zhang Xiaojuan Qi MQ 89 19 0 23 May 2024
Mixtral of Experts Albert Q. Jiang Alexandre Sablayrolles Antoine Roux A. Mensch Blanche Savary ... Théophile Gervet Thibaut Lavril Thomas Wang Timothée Lacroix William El Sayed MoE LLMAG 132 1,075 0 08 Jan 2024
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model BigScience Workshop : Teven Le Scao Angela Fan Christopher Akiki ... Zhongli Xie Zifan Ye M. Bras Younes Belkada Thomas Wolf VLM 330 2,377 0 09 Nov 2022
Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning Elias Frantar Sidak Pal Singh Dan Alistarh MQ 68 233 0 24 Aug 2022
BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction Yuhang Li Ruihao Gong Xu Tan Yang Yang Peng Hu Qi Zhang F. Yu Wei Wang Shi Gu MQ 99 433 0 10 Feb 2021
Up or Down? Adaptive Rounding for Post-Training Quantization Markus Nagel Rana Ali Amjad M. V. Baalen Christos Louizos Tijmen Blankevoort MQ 58 575 0 22 Apr 2020
Performance Guaranteed Network Acceleration via High-Order Residual Quantization Zefan Li Bingbing Ni Wenjun Zhang Xiaokang Yang Wen Gao MQ 58 107 0 29 Aug 2017
Balanced Quantization: An Effective and Efficient Approach to Quantized Neural Networks Shuchang Zhou Yuzhi Wang He Wen Qinyao He Yuheng Zou MQ 77 110 0 22 Jun 2017
Pointer Sentinel Mixture Models Stephen Merity Caiming Xiong James Bradbury R. Socher RALM 252 2,842 0 26 Sep 2016