Rethinking Channel Dimensions to Isolate Outliers for Low-bit Weight Quantization of Large Language Models

Rethinking Channel Dimensions to Isolate Outliers for Low-bit Weight Quantization of Large Language Models

27 September 2023

Jung Hwan Heo

Jeonghoon Kim

Beomseok Kwon

Byeongwook Kim

Se Jung Kwon

Dongsoo Lee

Papers citing "Rethinking Channel Dimensions to Isolate Outliers for Low-bit Weight Quantization of Large Language Models"

8 / 8 papers shown

Title
Semantic Retention and Extreme Compression in LLMs: Can We Have Both? Stanislas Laborde Martin Cousseau Antoun Yaacoub Lionel Prevost MQ 23 0 0 12 May 2025
SpikeLLM: Scaling up Spiking Neural Network to Large Language Models via Saliency-based Spiking Xingrun Xing Boyan Gao Zheng Zhang David A. Clifton Shitao Xiao LI DU Guoqi Li Jiajun Zhang 50 5 0 05 Jul 2024
LoQT: Low Rank Adapters for Quantized Training Sebastian Loeschcke M. Toftrup M. Kastoryano Serge J. Belongie Vésteinn Snæbjarnarson MQ 34 3 0 26 May 2024
No Token Left Behind: Reliable KV Cache Compression via Importance-Aware Mixed Precision Quantization J. Yang Byeongwook Kim Jeongin Bae Beomseok Kwon Gunho Park Eunho Yang S. Kwon Dongsoo Lee MQ 34 45 0 28 Feb 2024
L4Q: Parameter Efficient Quantization-Aware Fine-Tuning on Large Language Models Hyesung Jeon Yulhwa Kim Jae-Joon Kim MQ 24 4 0 07 Feb 2024
Memory-Efficient Fine-Tuning of Compressed Large Language Models via sub-4-bit Integer Quantization Jeonghoon Kim J. H. Lee Sungdong Kim Joonsuk Park Kang Min Yoo S. Kwon Dongsoo Lee MQ 41 98 0 23 May 2023
Training-Free Acceleration of ViTs with Delayed Spatial Merging J. Heo Seyedarmin Azizi A. Fayyazi Massoud Pedram 36 3 0 04 Mar 2023
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 253 1,989 0 31 Dec 2020