Does quantization affect models' performance on long-context tasks?

v1v2 (latest)

Does quantization affect models' performance on long-context tasks?

26 May 2025

Anirudh Atmakuru

Marzena Karpinska

ArXiv (abs)PDF HTML

Papers citing "Does quantization affect models' performance on long-context tasks?"

6 / 6 papers shown

Title
Through a Compressed Lens: Investigating the Impact of Quantization on LLM Explainability and Interpretability Qianli Wang Mingyang Wang Nils Feldhus Simon Ostermann Yuan Cao Hinrich Schütze Sebastian Möller Vera Schmitt MQ 60 1 0 20 May 2025
An Empirical Study of Qwen3 Quantization Xingyu Zheng Yuye Li Haoran Chu Yue Feng Xudong Ma Jie Luo Jinyang Guo Haotong Qin Michele Magno Xianglong Liu MQ 86 6 0 04 May 2025
Energy Considerations of Large Language Model Inference and Efficiency Optimizations Jared Fernandez Clara Na Vashisth Tiwari Yonatan Bisk Sasha Luccioni Emma Strubell 115 5 0 24 Apr 2025
Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Models Ruikang Liu Yuxuan Sun Manyi Zhang Haoli Bai Xianzhi Yu Tiezheng Yu C. Yuan Lu Hou MQ LRM 126 11 0 07 Apr 2025
Quantization Meets Reasoning: Exploring LLM Low-Bit Quantization Degradation for Mathematical Reasoning Zhen Li Yupeng Su Runming Yang C. Xie Zehua Wang Zhongwei Xie Ngai Wong Hongxia Yang MQ LRM 181 4 0 06 Jan 2025
An empirical study of LLaMA3 quantization: from LLMs to MLLMs Wei Huang Xingyu Zheng Xudong Ma Haotong Qin Chengtao Lv Hong Chen Jie Luo Xiaojuan Qi Xianglong Liu Michele Magno MQ 152 42 0 22 Apr 2024