GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers

31 October 2022

Dan Alistarh

Papers citing "GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers"

50 / 177 papers shown

Title
MoE-CAP: Benchmarking Cost, Accuracy and Performance of Sparse Mixture-of-Experts Systems Yinsicheng Jiang Yao Fu Yeqi Huang Ping Nie Zhan Lu ... Dayou Du Tairan Xu Kai Zou Edoardo Ponti Luo Mai MoE 17 0 0 16 May 2025
Insights into DeepSeek-V3: Scaling Challenges and Reflections on Hardware for AI Architectures Chenggang Zhao Chengqi Deng Chong Ruan Damai Dai Huazuo Gao ... Wenfeng Liang Ying He Yuping Wang Yuxuan Liu Y. X. Wei MoE 41 0 0 14 May 2025
Resource-Efficient Language Models: Quantization for Fast and Accessible Inference Tollef Emil Jørgensen MQ 54 0 0 13 May 2025
Semantic Retention and Extreme Compression in LLMs: Can We Have Both? Stanislas Laborde Martin Cousseau Antoun Yaacoub Lionel Prevost MQ 23 0 0 12 May 2025
Making Small Language Models Efficient Reasoners: Intervention, Supervision, Reinforcement Xuechen Zhang Zijian Huang Chenshun Ni Ziyang Xiong J. Chen Samet Oymak ReLM LRM 45 0 0 12 May 2025
GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidance Jinuk Kim Marwa El Halabi W. Park Clemens JS Schaefer Deokjae Lee Yeonhong Park Jae W. Lee Hyun Oh Song MQ 34 0 0 11 May 2025
Challenging GPU Dominance: When CPUs Outperform for On-Device LLM Inference Haolin Zhang Jeff Huang 35 0 0 09 May 2025
MxMoE: Mixed-precision Quantization for MoE with Accuracy and Performance Co-Design Haojie Duanmu Xiuhong Li Zhihang Yuan Size Zheng Jiangfei Duan Xingcheng Zhang Dahua Lin MQ MoE 165 0 0 09 May 2025
Diffusion Model Quantization: A Review Qian Zeng Chenggong Hu Mingli Song Jie Song MQ 45 0 0 08 May 2025
Optimizing LLMs for Resource-Constrained Environments: A Survey of Model Compression Techniques Sanjay Surendranath Girija Shashank Kapoor Lakshit Arora Dipen Pradhan Aman Raj Ankit Shetgaonkar 57 0 0 05 May 2025
An Empirical Study of Qwen3 Quantization Xingyu Zheng Yuye Li Haoran Chu Yue Feng Xudong Ma Jie Luo Jinyang Guo Haotong Qin Michele Magno Xianglong Liu MQ 29 0 0 04 May 2025
Efficient Fine-Tuning of Quantized Models via Adaptive Rank and Bitwidth Changhai Zhou Yuhua Zhou Qian Qiao Weizhong Zhang Cheng Jin MQ 27 0 0 02 May 2025
MoEQuant: Enhancing Quantization for Mixture-of-Experts Large Language Models via Expert-Balanced Sampling and Affinity Guidance Xing Hu Zhixuan Chen Dawei Yang Zukang Xu Chen Xu Zhihang Yuan Sifan Zhou Jiangyong Yu MoE MQ 41 0 0 02 May 2025
Scaling On-Device GPU Inference for Large Generative Models Jiuqiang Tang Raman Sarokin Ekaterina Ignasheva Grant Jensen Lin Chen Juhyun Lee Andrei Kulik Matthias Grundmann 123 1 0 01 May 2025
ICQuant: Index Coding enables Low-bit LLM Quantization Xinlin Li Osama A. Hanna Christina Fragouli Suhas Diggavi MQ 62 0 0 01 May 2025
Pushing the Limits of Low-Bit Optimizers: A Focus on EMA Dynamics Cong Xu Wenbin Liang Mo Yu Anan Liu Kaipeng Zhang Lizhuang Ma Yufei Guo Jun Wang Wenqi Zhang MQ 57 0 0 01 May 2025
FineScope : Precision Pruning for Domain-Specialized Large Language Models Using SAE-Guided Self-Data Cultivation Chaitali Bhattacharyya Yeseong Kim 45 0 0 01 May 2025
Precision Where It Matters: A Novel Spike Aware Mixed-Precision Quantization Strategy for LLaMA-based Language Models Lucas Maisonnave Cyril Moineau Olivier Bichler Fabrice Rastello MQ 71 1 0 30 Apr 2025
Legilimens: Performant Video Analytics on the System-on-Chip Edge M. Ramanujam Yinwei Dai Kyle Jamieson Ravi Netravali 78 0 0 29 Apr 2025
Softpick: No Attention Sink, No Massive Activations with Rectified Softmax Zayd Muhammad Kawakibi Zuhri Erland Hilman Fuadi Alham Fikri Aji 33 0 0 29 Apr 2025
BrAIcht, a theatrical agent that speaks like Bertolt Brecht's characters Baz Roland Kristina Malyseva Anna Pappa Tristan Cazenave 66 0 0 29 Apr 2025
Model Connectomes: A Generational Approach to Data-Efficient Language Models Klemen Kotar Greta Tuckute 51 0 0 29 Apr 2025
R-Sparse: Rank-Aware Activation Sparsity for Efficient LLM Inference Zhenyu (Allen) Zhang Zechun Liu Yuandong Tian Harshit Khaitan Zhilin Wang Steven Li 57 0 0 28 Apr 2025
TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate A. Zandieh Majid Daliri Majid Hadian Vahab Mirrokni MQ 74 0 0 28 Apr 2025
FineQ: Software-Hardware Co-Design for Low-Bit Fine-Grained Mixed-Precision Quantization of LLMs Xilong Xie Liang Wang Limin Xiao Meng Han Lin Sun S. Zheng Xiangrong Xu MQ 31 0 0 28 Apr 2025
PARD: Accelerating LLM Inference with Low-Cost PARallel Draft Model Adaptation Zihao An Huajun Bai Ziqiang Liu Dong Li E. Barsoum 61 0 0 23 Apr 2025
CAPO: Cost-Aware Prompt Optimization Tom Zehle Moritz Schlager Timo Heiß Matthias Feurer VLM 59 0 0 22 Apr 2025
Hexcute: A Tile-based Programming Language with Automatic Layout and Task-Mapping Synthesis Xiaotian Zhang Yaoyao Ding Yang Hu Gennady Pekhimenko 41 0 0 22 Apr 2025
A LoRA-Based Approach to Fine-Tuning LLMs for Educational Guidance in Resource-Constrained Settings Md Millat Hosen 25 0 0 22 Apr 2025
Gradual Binary Search and Dimension Expansion : A general method for activation quantization in LLMs Lucas Maisonnave Cyril Moineau Olivier Bichler Fabrice Rastello MQ 42 0 0 18 Apr 2025
From Large to Super-Tiny: End-to-End Optimization for Cost-Efficient LLMs Jiliang Ni Jiachen Pu Zhongyi Yang Kun Zhou Hui Wang Xiaoliang Xiao Dakui Wang Xin Li Jingfeng Luo Conggang Hu 37 0 0 18 Apr 2025
Tilus: A Virtual Machine for Arbitrary Low-Precision GPGPU Computation in LLM Serving Yaoyao Ding Bohan Hou X. Zhang Allan Lin Tianqi Chen Cody Yu Hao Yida Wang Gennady Pekhimenko 50 0 0 17 Apr 2025
70% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float Tianyi Zhang Yang Sui Shaochen Zhong V. Chaudhary Xia Hu Anshumali Shrivastava MQ 32 0 0 15 Apr 2025
Efficient Reasoning Models: A Survey Sicheng Feng Gongfan Fang Xinyin Ma Xinchao Wang ReLM LRM 148 2 0 15 Apr 2025
Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantization Yamato Arai Yuma Ichikawa MQ 34 0 0 13 Apr 2025
DL-QAT: Weight-Decomposed Low-Rank Quantization-Aware Training for Large Language Models Wenjin Ke Zhe Li D. Li Lu Tian E. Barsoum MQ 35 1 0 12 Apr 2025
Leveraging LLMs for Utility-Focused Annotation: Reducing Manual Effort for Retrieval and RAG Hengran Zhang Minghao Tang Keping Bi J. Guo Shihao Liu Daiting Shi Dawei Yin Xueqi Cheng 19 0 0 07 Apr 2025
Saliency-driven Dynamic Token Pruning for Large Language Models Yao Tao Yehui Tang Yun Wang Mingjian Zhu Hailin Hu Yunhe Wang 36 0 0 06 Apr 2025
Entropy-Based Block Pruning for Efficient Large Language Models Liangwei Yang Yuhui Xu Juntao Tan Doyen Sahoo Siyang Song Caiming Xiong David W. Romero Shelby Heinecke AAML 28 0 0 04 Apr 2025
GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibration Yuhang Li Ruokai Yin Donghyun Lee Shiting Xiao Priyadarshini Panda MQ 51 0 0 03 Apr 2025
Quamba2: A Robust and Scalable Post-training Quantization Framework for Selective State Space Models Hung-Yueh Chiang Chi-chih Chang N. Frumkin Kai-Chiang Wu Mohamed S. Abdelfattah Diana Marculescu MQ 146 0 0 28 Mar 2025
MoQa: Rethinking MoE Quantization with Multi-stage Data-model Distribution Awareness Zihao Zheng Xiuping Cui Size Zheng Maoliang Li Jiayu Chen Yun Liang Xiang Chen MQ MoE 64 0 0 27 Mar 2025
TopV: Compatible Token Pruning with Inference Time Optimization for Fast and Low-Memory Multimodal Vision Language Model Cheng Yang Yang Sui Jinqi Xiao Lingyi Huang Yu Gong ... Jinghua Yan Y. Bai P. Sadayappan Xia Hu Bo Yuan VLM 64 0 0 24 Mar 2025
Oaken: Fast and Efficient LLM Serving with Online-Offline Hybrid KV Cache Quantization Minsu Kim Seongmin Hong RyeoWook Ko S. Choi Hunjong Lee Junsoo Kim Joo-Young Kim Jongse Park 57 0 0 24 Mar 2025
Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models Yang Sui Yu-Neng Chuang Guanchu Wang Jiamu Zhang Tianyi Zhang ... Hongyi Liu Andrew Wen Shaochen Zhong Hanjie Chen OffRL ReLM LRM 83 31 0 20 Mar 2025
EfficientLLaVA:Generalizable Auto-Pruning for Large Vision-language Models Yinan Liang Zehua Wang Xiuwei Xu Jie Zhou Jiwen Lu VLM LRM 51 0 0 19 Mar 2025
ConSCompF: Consistency-focused Similarity Comparison Framework for Generative Large Language Models Alexey Karev Dong Xu 58 0 0 18 Mar 2025
Triad: Empowering LMM-based Anomaly Detection with Vision Expert-guided Visual Tokenizer and Manufacturing Process Yuanze Li Shihao Yuan Haolin Wang Qizhang Li Ming-Yu Liu Chen Xu Guangming Shi Wangmeng Zuo 56 0 0 17 Mar 2025
PLM: Efficient Peripheral Language Models Hardware-Co-Designed for Ubiquitous Computing Cheng Deng Luoyang Sun Jiwen Jiang Yongcheng Zeng Xinjian Wu ... Haoyang Li Lei Chen Lionel M. Ni Hongzhi Zhang Jun Wang 171 0 0 15 Mar 2025
PaCA: Partial Connection Adaptation for Efficient Fine-Tuning Sunghyeon Woo Sol Namkung Sunwoo Lee Inho Jeong Beomseok Kim Dongsuk Jeon 39 0 0 28 Feb 2025