AsymKV: Enabling 1-Bit Quantization of KV Cache with Layer-Wise Asymmetric Quantization Configurations

17 October 2024

Papers citing "AsymKV: Enabling 1-Bit Quantization of KV Cache with Layer-Wise Asymmetric Quantization Configurations"

3 / 3 papers shown

Title
GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLM Hao Kang Qingru Zhang Souvik Kundu Geonhwa Jeong Zaoxing Liu Tushar Krishna Tuo Zhao MQ 120 87 0 08 Mar 2024
IntactKV: Improving Large Language Model Quantization by Keeping Pivot Tokens Intact Ruikang Liu Haoli Bai Haokun Lin Yuening Li Han Gao Zheng-Jun Xu Lu Hou Jun Yao Chun Yuan MQ 28 29 0 02 Mar 2024
No Token Left Behind: Reliable KV Cache Compression via Importance-Aware Mixed Precision Quantization J. Yang Byeongwook Kim Jeongin Bae Beomseok Kwon Gunho Park Eunho Yang S. Kwon Dongsoo Lee MQ 87 49 0 28 Feb 2024