ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers

4 June 2022

Z. Yao

Reza Yazdani Aminabadi

Yuxiong He

Papers citing "ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers"

50 / 324 papers shown

Title
Accurate KV Cache Quantization with Outlier Tokens Tracing Yi Su Yuechi Zhou Quantong Qiu Juntao Li Qingrong Xia Ping Li Xinyu Duan Zhefeng Wang Min Zhang MQ 22 0 0 16 May 2025
Diffusion Model Quantization: A Review Qian Zeng Chenggong Hu Mingli Song Jie Song MQ 45 0 0 08 May 2025
Optimizing LLMs for Resource-Constrained Environments: A Survey of Model Compression Techniques Sanjay Surendranath Girija Shashank Kapoor Lakshit Arora Dipen Pradhan Aman Raj Ankit Shetgaonkar 57 0 0 05 May 2025
EntroLLM: Entropy Encoded Weight Compression for Efficient Large Language Model Inference on Edge Devices Arnab Sanyal Prithwish Mukherjee Gourav Datta Sandeep P. Chinchali MQ 131 0 0 05 May 2025
Radio: Rate-Distortion Optimization for Large Language Model Compression Sean I. Young MQ 23 0 0 05 May 2025
Efficient Fine-Tuning of Quantized Models via Adaptive Rank and Bitwidth Changhai Zhou Yuhua Zhou Qian Qiao Weizhong Zhang Cheng Jin MQ 27 0 0 02 May 2025
FineQ: Software-Hardware Co-Design for Low-Bit Fine-Grained Mixed-Precision Quantization of LLMs Xilong Xie Liang Wang Limin Xiao Meng Han Lin Sun S. Zheng Xiangrong Xu MQ 31 0 0 28 Apr 2025
Bi-directional Model Cascading with Proxy Confidence David Warren Mark Dras 46 0 0 27 Apr 2025
L3: DIMM-PIM Integrated Architecture and Coordination for Scalable Long-Context LLM Inference Qingyuan Liu Liyan Chen Yanning Yang Haoyu Wang Dong Du Zhigang Mao Naifeng Jing Yubin Xia Haibo Chen 36 0 0 24 Apr 2025
Towards Harnessing the Collaborative Power of Large and Small Models for Domain Tasks Yang Liu Bingjie Yan Tianyuan Zou Jianqing Zhang Zixuan Gu ... Jiajian Li Xiaozhou Ye Ye Ouyang Qiang Yang Wenjie Qu ALM 155 1 0 24 Apr 2025
Compute-Optimal LLMs Provably Generalize Better With Scale Marc Finzi Sanyam Kapoor Diego Granziol Anming Gu Christopher De Sa J. Zico Kolter Andrew Gordon Wilson 30 0 0 21 Apr 2025
NoWag: A Unified Framework for Shape Preserving Compression of Large Language Models Lawrence Liu Inesh Chakrabarti Yixiao Li Mengdi Wang Tuo Zhao Lin F. Yang MQ 33 0 0 20 Apr 2025
Collaborative Learning of On-Device Small Model and Cloud-Based Large Model: Advances and Future Directions Chaoyue Niu Yucheng Ding Junhui Lu Zhengxiang Huang Hang Zeng Yutong Dai Xuezhen Tu Chengfei Lv Fan Wu Guihai Chen 32 1 0 17 Apr 2025
Accelerating Clinical NLP at Scale with a Hybrid Framework with Reduced GPU Demands: A Case Study in Dementia Identification Jianlin Shi Qiwei Gan Elizabeth Hanchrow Annie Bowles John Stanley Adam P. Bress Jordana B. Cohen Patrick R. Alba 32 0 0 16 Apr 2025
TAMP: Token-Adaptive Layerwise Pruning in Multimodal Large Language Models Jaewoo Lee Keyang Xuan Chanakya Ekbote Sandeep Polisetty Yi Ren Fung Paul Pu Liang VLM 37 0 0 14 Apr 2025
Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantization Yamato Arai Yuma Ichikawa MQ 34 0 0 13 Apr 2025
End-to-End Dialog Neural Coreference Resolution: Balancing Efficiency and Accuracy in Large-Scale Systems Zhang Dong Songhang deng Mingbang Wang Le Dai Jiyuan Li Xingzu Liu Ruilin Nong 21 0 0 08 Apr 2025
Sustainable LLM Inference for Edge AI: Evaluating Quantized LLMs for Energy Efficiency, Output Accuracy, and Inference Latency E. J. Husom Arda Goknil Merve Astekin Lwin Khin Shar Andre Kåsen S. Sen Benedikt Andreas Mithassel Ahmet Soylu MQ 43 0 0 04 Apr 2025
MiLo: Efficient Quantized MoE Inference with Mixture of Low-Rank Compensators Beichen Huang Yueming Yuan Zelei Shao Minjia Zhang MQ MoE 42 0 0 03 Apr 2025
When Reasoning Meets Compression: Benchmarking Compressed Large Reasoning Models on Complex Reasoning Tasks Nan Zhang Yusen Zhang Prasenjit Mitra Rui Zhang MQ LRM 59 2 0 02 Apr 2025
SQuat: Subspace-orthogonal KV Cache Quantization Hao Wang Ligong Han Kai Xu Akash Srivastava MQ 51 0 0 31 Mar 2025
Model Hemorrhage and the Robustness Limits of Large Language Models Ziyang Ma Zehan Li L. Zhang Gui-Song Xia Bo Du Liangpei Zhang Dacheng Tao 59 0 0 31 Mar 2025
Think Before Recommend: Unleashing the Latent Reasoning Power for Sequential Recommendation Jiakai Tang Sunhao Dai Teng Shi Jun Xu X. Chen Wen Chen Wu Jian Yuning Jiang LRM 75 5 0 28 Mar 2025
Oaken: Fast and Efficient LLM Serving with Online-Offline Hybrid KV Cache Quantization Minsu Kim Seongmin Hong RyeoWook Ko S. Choi Hunjong Lee Junsoo Kim Joo-Young Kim Jongse Park 57 0 0 24 Mar 2025
PARQ: Piecewise-Affine Regularized Quantization Lisa Jin Jianhao Ma Zechun Liu Andrey Gromov Aaron Defazio Lin Xiao MQ 43 0 0 19 Mar 2025
FP4DiT: Towards Effective Floating Point Quantization for Diffusion Transformers Ruichen Chen Keith G. Mills Di Niu MQ 56 0 0 19 Mar 2025
Striving for Simplicity: Simple Yet Effective Prior-Aware Pseudo-Labeling for Semi-Supervised Ultrasound Image Segmentation Yaxiong Chen Yujie Wang Zixuan Zheng Jingliang Hu Yilei Shi Shengwu Xiong Xiao Xiang Zhu Lichao Mou 54 1 0 18 Mar 2025
Accurate INT8 Training Through Dynamic Block-Level Fallback Pengle Zhang Jia wei Jintao Zhang Jun-Jie Zhu Jianfei Chen MQ 82 3 0 13 Mar 2025
SEAP: Training-free Sparse Expert Activation Pruning Unlock the Brainpower of Large Language Models Xun Liang Hanyu Wang Huayi Lai Simin Niu Shichao Song Jiawei Yang Jihao Zhao Feiyu Xiong Bo Tang Z. Li VLM 47 0 0 10 Mar 2025
SplitQuantV2: Enhancing Low-Bit Quantization of LLMs Without GPUs Jaewoo Song Fangzhen Lin MQ 51 0 0 07 Mar 2025
MergeQuant: Accurate 4-bit Static Quantization of Large Language Models by Channel-wise Calibration Jinguang Wang Yufei Guo Haifeng Sun Tingting Yang Zirui Zhuang Wanyi Ning Yuexi Yin Q. Qi Jianxin Liao MQ MoMe 51 0 0 07 Mar 2025
CASP: Compression of Large Multimodal Models Based on Attention Sparsity Mohsen Gholami Mohammad Akbari Kevin Cannons Yong Zhang 65 0 0 07 Mar 2025
QArtSR: Quantization via Reverse-Module and Timestep-Retraining in One-Step Diffusion based Image Super-Resolution Libo Zhu Haotong Qin Kaicheng Yang W. J. Li Yong Guo Yulun Zhang Susanto Rahardja Xiaokang Yang MQ DiffM 66 0 0 07 Mar 2025
Universality of Layer-Level Entropy-Weighted Quantization Beyond Model Architecture and Size Alireza Behtash Marijan Fofonjka Ethan Baird Tyler Mauer Hossein Moghimifam David Stout Joel Dennison MQ 58 1 0 06 Mar 2025
RSQ: Learning from Important Tokens Leads to Better Quantized LLMs Yi-Lin Sung Prateek Yadav Jialu Li Jaehong Yoon Joey Tianyi Zhou MQ 54 1 0 03 Mar 2025
HALO: Hardware-aware quantization with low critical-path-delay weights for LLM acceleration Rohan Juneja Shivam Aggarwal Safeen Huda Tulika Mitra L. Peh 50 0 0 27 Feb 2025
Binary Neural Networks for Large Language Model: A Survey Liangdong Liu Zhitong Zheng Cong Wang TianHuang Su ZhenYu Yang MQ 67 0 0 26 Feb 2025
The Lottery LLM Hypothesis, Rethinking What Abilities Should LLM Compression Preserve? Zhenheng Tang Xiang Liu Qian Wang Peijie Dong Bingsheng He Xiaowen Chu Bo Li LRM 61 1 0 24 Feb 2025
More for Keys, Less for Values: Adaptive KV Cache Quantization Mohsen Hariri Lam Nguyen Sixu Chen Shaochen Zhong Qifan Wang Xia Hu Xiaotian Han V. Chaudhary MQ 48 0 0 24 Feb 2025
When Compression Meets Model Compression: Memory-Efficient Double Compression for Large Language Models Weilan Wang Yu Mao Dongdong Tang Hongchao Du Nan Guan Chun Jason Xue MQ 67 1 0 24 Feb 2025
KVLink: Accelerating Large Language Models via Efficient KV Cache Reuse Jingbo Yang Bairu Hou Wei Wei Yujia Bao Shiyu Chang VLM 41 2 0 21 Feb 2025
FedSpaLLM: Federated Pruning of Large Language Models Guangji Bai Yijiang Li Zilinghan Li Liang Zhao Kibaek Kim FedML 65 4 0 20 Feb 2025
EvoP: Robust LLM Inference via Evolutionary Pruning Shangyu Wu Hongchao Du Ying Xiong Shuai Chen Tei-Wei Kuo Nan Guan Chun Jason Xue 34 1 0 19 Feb 2025
Benchmarking Post-Training Quantization in LLMs: Comprehensive Taxonomy, Unified Evaluation, and Comparative Analysis J. Zhao Hao Wu Miao Zhang Yuzhang Shang Xuebo Liu Yaowei Wang Min Zhang Liqiang Nie MQ 60 1 0 18 Feb 2025
QuZO: Quantized Zeroth-Order Fine-Tuning for Large Language Models Jiajun Zhou Yifan Yang Kai Zhen Z. Liu Yequan Zhao Ershad Banijamali Athanasios Mouchtaris Ngai Wong Zheng Zhang MQ 41 0 0 17 Feb 2025
Efficient Long-Decoding Inference with Reasoning-Aware Attention Sparsity Junhao Hu Wenrui Huang Weidong Wang Zhenwen Li Tiancheng Hu Zhixia Liu Xusheng Chen Tao Xie Yizhou Shan LRM 51 0 0 16 Feb 2025
Lossless Acceleration of Large Language Models with Hierarchical Drafting based on Temporal Locality in Speculative Decoding Sukmin Cho S. Choi T. Hwang Jeongyeon Seo Soyeong Jeong Huije Lee Hoyun Song Jong C. Park Youngjin Kwon 51 0 0 08 Feb 2025
BCQ: Block Clustered Quantization for 4-bit (W4A4) LLM Inference Reena Elangovan Charbel Sakr A. Raghunathan Brucek Khailany MQ 50 1 0 07 Feb 2025
M2R2: Mixture of Multi-Rate Residuals for Efficient Transformer Inference Nikhil Bhendawade Mahyar Najibi Devang Naik Irina Belousova MoE 85 0 0 04 Feb 2025
Progressive Binarization with Semi-Structured Pruning for LLMs Xinyu Yan Tianao Zhang Zhiteng Li Yulun Zhang MQ 54 0 0 03 Feb 2025