LLM-QAT: Data-Free Quantization Aware Training for Large Language Models

LLM-QAT: Data-Free Quantization Aware Training for Large Language Models

29 May 2023

Changsheng Zhao

Yashar Mehdad

Raghuraman Krishnamoorthi

ArXiv (abs)PDF HTML

Papers citing "LLM-QAT: Data-Free Quantization Aware Training for Large Language Models"

8 / 58 papers shown

Title
Compressing LLMs: The Truth is Rarely Pure and Never Simple Ajay Jaiswal Zhe Gan Xianzhi Du Bowen Zhang Zhangyang Wang Yinfei Yang MQ 128 50 0 02 Oct 2023
Rethinking Channel Dimensions to Isolate Outliers for Low-bit Weight Quantization of Large Language Models Jung Hwan Heo Jeonghoon Kim Beomseok Kwon Byeongwook Kim Se Jung Kwon Dongsoo Lee MQ 129 10 0 27 Sep 2023
Understanding the Impact of Post-Training Quantization on Large Language Models Somnath Roy MQ 77 5 0 11 Sep 2023
Norm Tweaking: High-performance Low-bit Quantization of Large Language Models Liang Li Qingyuan Li Bo Zhang Xiangxiang Chu MQ 105 34 0 06 Sep 2023
FPTQ: Fine-grained Post-Training Quantization for Large Language Models Qingyuan Li Yifan Zhang Liang Li Peng Yao Bo Zhang Xiangxiang Chu Yerui Sun Li-Qiang Du Yuchen Xie MQ 108 13 0 30 Aug 2023
Do Emergent Abilities Exist in Quantized Large Language Models: An Empirical Study Peiyu Liu Zikang Liu Ze-Feng Gao Dawei Gao Wayne Xin Zhao Yaliang Li Bolin Ding Ji-Rong Wen MQ LRM 94 35 0 16 Jul 2023
SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networks Rui-Jie Zhu Qihang Zhao Guoqi Li Jason K. Eshraghian BDL VLM 111 89 0 27 Feb 2023
Quantization Robust Federated Learning for Efficient Inference on Heterogeneous Devices Kartik Gupta Marios Fournarakis M. Reisser Christos Louizos Markus Nagel FedML 69 16 0 22 Jun 2022