OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models

25 August 2023

Yu Qiao

Ping Luo

ArXiv PDF HTML

Papers citing "OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models"

50 / 139 papers shown

Title
Semantic Retention and Extreme Compression in LLMs: Can We Have Both? Stanislas Laborde Martin Cousseau Antoun Yaacoub Lionel Prevost MQ 23 0 0 12 May 2025
Diffusion Model Quantization: A Review Qian Zeng Chenggong Hu Mingli Song Jie Song MQ 45 0 0 08 May 2025
Radio: Rate-Distortion Optimization for Large Language Model Compression Sean I. Young MQ 23 0 0 05 May 2025
RWKVQuant: Quantizing the RWKV Family with Proxy Guided Hybrid of Scalar and Vector Quantization Chen Xu Yuxuan Yue Zukang Xu Xing Hu Jiangyong Yu Zhixuan Chen Sifan Zhou Zhihang Yuan Dawei Yang MQ 29 0 0 02 May 2025
Fast and Low-Cost Genomic Foundation Models via Outlier Removal Haozheng Luo Chenghao Qiu Maojiang Su Zhihan Zhou Zoe Mehta Guo Ye Jerry Yao-Chieh Hu Han Liu AAML 55 1 0 01 May 2025
ICQuant: Index Coding enables Low-bit LLM Quantization Xinlin Li Osama A. Hanna Christina Fragouli Suhas Diggavi MQ 62 0 0 01 May 2025
Enhancing Ultra-Low-Bit Quantization of Large Language Models Through Saliency-Aware Partial Retraining Deyu Cao Samin Aref MQ 27 0 0 14 Apr 2025
Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantization Yamato Arai Yuma Ichikawa MQ 34 0 0 13 Apr 2025
Task-Circuit Quantization: Leveraging Knowledge Localization and Interpretability for Compression Hanqi Xiao Yi-Lin Sung Elias Stengel-Eskin Joey Tianyi Zhou MQ 38 0 0 10 Apr 2025
AccLLM: Accelerating Long-Context LLM Inference Via Algorithm-Hardware Co-Design Yanbiao Liang Huihong Shi Haikuo Shao Zhongfeng Wang 33 0 0 07 Apr 2025
GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibration Yuhang Li Ruokai Yin Donghyun Lee Shiting Xiao Priyadarshini Panda MQ 51 0 0 03 Apr 2025
LLMPi: Optimizing LLMs for High-Throughput on Raspberry Pi Mahsa Ardakani Jinendra Malekar Ramtin Zand MQ 42 0 0 02 Apr 2025
When Reasoning Meets Compression: Benchmarking Compressed Large Reasoning Models on Complex Reasoning Tasks Nan Zhang Yusen Zhang Prasenjit Mitra Rui Zhang MQ LRM 59 2 0 02 Apr 2025
PIM-LLM: A High-Throughput Hybrid PIM Architecture for 1-bit LLMs Jinendra Malekar Peyton S. Chandarana Md Hasibul Amin Mohammed E. Elbtity Ramtin Zand 26 1 0 31 Mar 2025
RaanA: A Fast, Flexible, and Data-Efficient Post-Training Quantization Algorithm Yongyi Yang Jianyang Gao Wei Hu MQ 36 1 0 29 Mar 2025
QUAD: Quantization and Parameter-Efficient Tuning of LLM with Activation Decomposition Yuxuan Hu Xiaodong Chen C. Li Hongyu Chen J. Zhang MQ 60 0 0 25 Mar 2025
PIPO: Pipelined Offloading for Efficient Inference on Consumer Devices Yangyijian Liu Jun Yu Li Wu-Jun Li 31 0 0 15 Mar 2025
ViM-VQ: Efficient Post-Training Vector Quantization for Visual Mamba Juncan Deng Shuaiting Li Zeyu Wang Kedong Xu Hong Gu Kejie Huang MQ 60 0 0 12 Mar 2025
Post-Training Quantization for Diffusion Transformer via Hierarchical Timestep Grouping Ning Ding Jing Han Yuchuan Tian Chao Xu Kai Han Yehui Tang MQ 44 0 0 10 Mar 2025
DistiLLM-2: A Contrastive Approach Boosts the Distillation of LLMs Jongwoo Ko Tianyi Chen Sungnyun Kim Tianyu Ding Luming Liang Ilya Zharkov Se-Young Yun VLM 171 0 0 10 Mar 2025
SAQ-SAM: Semantically-Aligned Quantization for Segment Anything Model Jing Zhang Zeju Li Qingyi Gu MQ VLM 56 0 0 09 Mar 2025
MergeQuant: Accurate 4-bit Static Quantization of Large Language Models by Channel-wise Calibration Jinguang Wang Yufei Guo Haifeng Sun Tingting Yang Zirui Zhuang Wanyi Ning Yuexi Yin Q. Qi Jianxin Liao MQ MoMe 51 0 0 07 Mar 2025
QArtSR: Quantization via Reverse-Module and Timestep-Retraining in One-Step Diffusion based Image Super-Resolution Libo Zhu Haotong Qin Kaicheng Yang W. J. Li Yong Guo Yulun Zhang Susanto Rahardja Xiaokang Yang MQ DiffM 66 0 0 07 Mar 2025
RSQ: Learning from Important Tokens Leads to Better Quantized LLMs Yi-Lin Sung Prateek Yadav Jialu Li Jaehong Yoon Joey Tianyi Zhou MQ 54 1 0 03 Mar 2025
Identifying Sensitive Weights via Post-quantization Integral Yuezhou Hu Weiyu Huang Zichen Liang Cheng Chen Jintao Zhang Jun Zhu Jianfei Chen MQ 47 2 0 28 Feb 2025
SpinQuant: LLM quantization with learned rotations Zechun Liu Changsheng Zhao Igor Fedorov Bilge Soran Dhruv Choudhary Raghuraman Krishnamoorthi Vikas Chandra Yuandong Tian Tijmen Blankevoort MQ 137 85 0 21 Feb 2025
BCQ: Block Clustered Quantization for 4-bit (W4A4) LLM Inference Reena Elangovan Charbel Sakr A. Raghunathan Brucek Khailany MQ 50 1 0 07 Feb 2025
QuantSpec: Self-Speculative Decoding with Hierarchical Quantized KV Cache Rishabh Tiwari Haocheng Xi Aditya Tomar Coleman Hooper Sehoon Kim Maxwell Horton Mahyar Najibi Michael W. Mahoney Kemal Kurniawan Amir Gholami MQ 64 1 0 05 Feb 2025
ParetoQ: Scaling Laws in Extremely Low-bit LLM Quantization Zechun Liu Changsheng Zhao Hanxian Huang Sijia Chen Jing Zhang ... Yuandong Tian Bilge Soran Raghuraman Krishnamoorthi Tijmen Blankevoort Vikas Chandra MQ 78 3 0 04 Feb 2025
PrefixQuant: Eliminating Outliers by Prefixed Tokens for Large Language Models Quantization Yonghong Tian Yi Liu Jiahao Wang Yi Bin Wenqi Shao Ping Luo MQ 63 2 0 28 Jan 2025
OstQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fitting Xing Hu Yuan Cheng Dawei Yang Zukang Xu Zhihang Yuan Jiangyong Yu Chen Xu Zhe Jiang Sifan Zhou MQ 41 5 0 23 Jan 2025
DiscQuant: A Quantization Method for Neural Networks Inspired by Discrepancy Theory Jerry Chee A. Backurs Rainie Heck Li Zhang Janardhan Kulkarni Thomas Rothvoss Sivakanth Gopi MQ 49 0 0 11 Jan 2025
Pushing the Envelope of Low-Bit LLM via Dynamic Error Compensation Y. Park Jake Hyun Hojoon Kim Jae W. Lee MQ 46 0 0 31 Dec 2024
LSAQ: Layer-Specific Adaptive Quantization for Large Language Model Deployment Binrui Zeng Bin Ji Xiaodong Liu Jie Yu Shasha Li Jun Ma Xiaopeng Li Shangwen Wang Xinran Hong Yongtao Tang MQ 42 1 0 24 Dec 2024
Deploying Foundation Model Powered Agent Services: A Survey Wenchao Xu Jinyu Chen Peirong Zheng Xiaoquan Yi Tianyi Tian ... Quan Wan Yining Qi Yunfeng Fan Qinliang Su Xuemin Shen AI4CE 119 1 0 18 Dec 2024
SKIM: Any-bit Quantization Pushing The Limits of Post-Training Quantization Runsheng Bai Qiang Liu B. Liu MQ 72 1 0 05 Dec 2024
Mixture of Cache-Conditional Experts for Efficient Mobile Device Inference Andrii Skliar T. V. Rozendaal Romain Lepert Todor Boinovski M. V. Baalen Markus Nagel Paul N. Whatmough B. Bejnordi MoE 84 1 0 27 Nov 2024
FAMES: Fast Approximate Multiplier Substitution for Mixed-Precision Quantized DNNs--Down to 2 Bits! Yi Ren Ruge Xu Xinfei Guo Weikang Qian MQ 74 0 0 27 Nov 2024
Anda: Unlocking Efficient LLM Inference with a Variable-Length Grouped Activation Data Format Chao Fang Man Shi Robin Geens Arne Symons Zhongfeng Wang Marian Verhelst 74 0 0 24 Nov 2024
AutoMixQ: Self-Adjusting Quantization for High Performance Memory-Efficient Fine-Tuning Changhai Zhou Shiyang Zhang Yuhua Zhou Zekai Liu Shichao Weng MQ 64 0 0 21 Nov 2024
BitMoD: Bit-serial Mixture-of-Datatype LLM Acceleration Yuzong Chen Ahmed F. AbouElhamayed Xilai Dai Yang Wang Marta Andronic George A. Constantinides Mohamed S. Abdelfattah MQ 108 1 0 18 Nov 2024
AMXFP4: Taming Activation Outliers with Asymmetric Microscaling Floating-Point for 4-bit LLM Inference Janghwan Lee Jiwoong Park Jinseok Kim Yongjik Kim Jungju Oh Jinwook Oh Jungwook Choi 44 2 0 15 Nov 2024
The Super Weight in Large Language Models Mengxia Yu De Wang Qi Shan Colorado Reed Alvin Wan MQ MILM 42 9 0 11 Nov 2024
Interactions Across Blocks in Post-Training Quantization of Large Language Models Khasmamad Shabanovi Lukas Wiest Vladimir Golkov Daniel Cremers Thomas Pfeil MQ 33 1 0 06 Nov 2024
BitStack: Any-Size Compression of Large Language Models in Variable Memory Environments Xinghao Wang Pengyu Wang Bo Wang Dong Zhang Yunhua Zhou Xipeng Qiu MQ 41 2 0 31 Oct 2024
LLMCBench: Benchmarking Large Language Model Compression for Efficient Deployment Ge Yang Changyi He J. Guo Jianyu Wu Yifu Ding Aishan Liu Haotong Qin Pengliang Ji Xianglong Liu MQ 33 4 0 28 Oct 2024
TesseraQ: Ultra Low-Bit LLM Post-Training Quantization with Block Reconstruction Yuhang Li Priyadarshini Panda MQ 31 1 0 24 Oct 2024
Beware of Calibration Data for Pruning Large Language Models Yixin Ji Yang Xiang Juntao Li Qingrong Xia Ping Li Xinyu Duan Zhefeng Wang Min Zhang 42 2 0 23 Oct 2024
DAQ: Density-Aware Post-Training Weight-Only Quantization For LLMs Yingsong Luo Ling Chen MQ 23 0 0 16 Oct 2024
Channel-Wise Mixed-Precision Quantization for Large Language Models Zihan Chen Bike Xie Jundong Li Cong Shen MQ 32 2 0 16 Oct 2024