ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Models

16 August 2024

Papers citing "ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Models"

7 / 7 papers shown

Title
Achieving binary weight and activation for LLMs using Post-Training Quantization Siqing Song Chuang Wang Ruiqi Wang Yi Yang Xuyao Zhang MQ 28 0 0 07 Apr 2025
Membership Inference Risks in Quantized Models: A Theoretical and Empirical Study Eric Aubinais Philippe Formont Pablo Piantanida Elisabeth Gassiat 50 0 0 10 Feb 2025
GQSA: Group Quantization and Sparsity for Accelerating Large Language Model Inference Chao Zeng Songwei Liu Shu Yang Fangmin Chen Xing Mei Lean Fu MQ 42 0 0 23 Dec 2024
SKIM: Any-bit Quantization Pushing The Limits of Post-Training Quantization Runsheng Bai Qiang Liu B. Liu MQ 72 1 0 05 Dec 2024
STBLLM: Breaking the 1-Bit Barrier with Structured Binary LLMs Peijie Dong Lujun Li Dayou Du Yuhan Chen Zhenheng Tang ... Wei Xue Wenhan Luo Qi-fei Liu Yi-Ting Guo Xiaowen Chu MQ 50 4 0 03 Aug 2024
QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks Albert Tseng Jerry Chee Qingyao Sun Volodymyr Kuleshov Christopher De Sa MQ 128 101 0 06 Feb 2024
BiLLM: Pushing the Limit of Post-Training Quantization for LLMs Wei Huang Yangdong Liu Haotong Qin Ying Li Shiming Zhang Xianglong Liu Michele Magno Xiaojuan Qi MQ 82 69 0 06 Feb 2024