ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers

4 June 2022

Z. Yao

Reza Yazdani Aminabadi

Yuxiong He

Papers citing "ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers"

50 / 324 papers shown

Title
Extreme Compression of Large Language Models via Additive Quantization Vage Egiazarian Andrei Panferov Denis Kuznedelev Elias Frantar Artem Babenko Dan Alistarh MQ 100 90 0 11 Jan 2024
Memory-Efficient Fine-Tuning for Quantized Diffusion Model Hyogon Ryu Seohyun Lim Hyunjung Shim DiffM MQ 27 6 0 09 Jan 2024
FlightLLM: Efficient Large Language Model Inference with a Complete Mapping Flow on FPGAs Shulin Zeng Jun Liu Guohao Dai Xinhao Yang Tianyu Fu ... Zehao Wang Ruoyu Zhang Kairui Wen Xuefei Ning Yu Wang 62 55 0 08 Jan 2024
Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systems Xupeng Miao Gabriele Oliaro Zhihao Zhang Xinhao Cheng Hongyi Jin Tianqi Chen Zhihao Jia 67 76 0 23 Dec 2023
Understanding the Potential of FPGA-Based Spatial Acceleration for Large Language Model Inference Hongzheng Chen Jiahao Zhang Yixiao Du Shaojie Xiang Zichao Yue Niansong Zhang Yaohui Cai Zhiru Zhang 55 34 0 23 Dec 2023
ConsistentEE: A Consistent and Hardness-Guided Early Exiting Method for Accelerating Language Models Inference Ziqian Zeng Yihuai Hong Hongliang Dai Huiping Zhuang Cen Chen 24 10 0 19 Dec 2023
An Adaptive Placement and Parallelism Framework for Accelerating RLHF Training Youshao Xiao Weichang Wu Zhenglei Zhou Fagui Mao Shangchun Zhao Lin Ju Lei Liang Xiaolu Zhang Jun Zhou 34 5 0 19 Dec 2023
TigerBot: An Open Multilingual Multitask LLM Ye Chen Wei Cai Liangming Wu Xiaowei Li Zhanxuan Xin Cong Fu 96 11 0 14 Dec 2023
ZeroQuant(4+2): Redefining LLMs Quantization with a New FP6-Centric Strategy for Diverse Generative Tasks Xiaoxia Wu Haojun Xia Stephen Youn Zhen Zheng Shiyang Chen ... Reza Yazdani Aminabadi Yuxiong He Olatunji Ruwase Leon Song Zhewei Yao 71 8 0 14 Dec 2023
CBQ: Cross-Block Quantization for Large Language Models Xin Ding Xiaoyu Liu Zhijun Tu Yun-feng Zhang Wei Li ... Hanting Chen Yehui Tang Zhiwei Xiong Baoqun Yin Yunhe Wang MQ 36 13 0 13 Dec 2023
EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelism Yanxi Chen Xuchen Pan Yaliang Li Bolin Ding Jingren Zhou LRM 41 31 0 08 Dec 2023
SmoothQuant+: Accurate and Efficient 4-bit Post-Training WeightQuantization for LLM Jiayi Pan Chengcan Wang Kaifu Zheng Yangguang Li Zhenyu Wang Bin Feng MQ 40 7 0 06 Dec 2023
Visual Prompting Upgrades Neural Network Sparsification: A Data-Model Perspective Can Jin Tianjin Huang Yihua Zhang Mykola Pechenizkiy Sijia Liu Shiwei Liu Tianlong Chen VLM 33 26 0 03 Dec 2023
The Efficiency Spectrum of Large Language Models: An Algorithmic Survey Tianyu Ding Tianyi Chen Haidong Zhu Jiachen Jiang Yiqi Zhong Jinxin Zhou Guangzhi Wang Zhihui Zhu Ilya Zharkov Luming Liang 29 22 0 01 Dec 2023
Nonparametric Variational Regularisation of Pretrained Transformers Fabio Fehr James Henderson 43 0 0 01 Dec 2023
LinguaLinked: A Distributed Large Language Model Inference System for Mobile Devices Junchen Zhao Yurun Song Simeng Liu Ian G. Harris S. Jyothi 26 5 0 01 Dec 2023
LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuning Han Guo P. Greengard Eric P. Xing Yoon Kim MQ 36 43 0 20 Nov 2023
HexGen: Generative Inference of Large Language Model over Heterogeneous Environment Youhe Jiang Ran Yan Xiaozhe Yao Yang Zhou Beidi Chen Binhang Yuan SyDa 30 10 0 20 Nov 2023
A Speed Odyssey for Deployable Quantization of LLMs Qingyuan Li Ran Meng Yiduo Li Bo-Wen Zhang Liang Li Yifan Lu Xiangxiang Chu Yerui Sun Yuchen Xie MQ 61 7 0 16 Nov 2023
Speculative Contrastive Decoding Hongyi Yuan Keming Lu Fei Huang Zheng Yuan Chang Zhou 39 5 0 15 Nov 2023
REST: Retrieval-Based Speculative Decoding Zhenyu He Zexuan Zhong Tianle Cai Jason D. Lee Di He RALM 19 76 0 14 Nov 2023
Enhancing Computation Efficiency in Large Language Models through Weight and Activation Quantization Jangwhan Lee Minsoo Kim Seungcheol Baek Seok Joong Hwang Wonyong Sung Jungwook Choi MQ 13 17 0 09 Nov 2023
Beyond Size: How Gradients Shape Pruning Decisions in Large Language Models Rocktim Jyoti Das Mingjie Sun Liqun Ma Zhiqiang Shen VLM 13 13 0 08 Nov 2023
Dissecting the Runtime Performance of the Training, Fine-tuning, and Inference of Large Language Models Longteng Zhang Xiang Liu Zeyu Li Xinglin Pan Peijie Dong ... Rui Guo Xin Wang Qiong Luo S. Shi Xiaowen Chu 43 7 0 07 Nov 2023
S-LoRA: Serving Thousands of Concurrent LoRA Adapters Ying Sheng Shiyi Cao Dacheng Li Coleman Hooper Nicholas Lee ... Banghua Zhu Lianmin Zheng Kurt Keutzer Joseph E. Gonzalez Ion Stoica MoE 28 88 0 06 Nov 2023
AFPQ: Asymmetric Floating Point Quantization for LLMs Yijia Zhang Sicheng Zhang Shijie Cao Dayou Du Jianyu Wei Ting Cao Ningyi Xu MQ 27 5 0 03 Nov 2023
AWEQ: Post-Training Quantization with Activation-Weight Equalization for Large Language Models Baisong Li Xingwang Wang Haixiao Xu MQ 22 0 0 02 Nov 2023
Atom: Low-bit Quantization for Efficient and Accurate LLM Serving Yilong Zhao Chien-Yu Lin Kan Zhu Zihao Ye Lequn Chen Wenlei Bao Luis Ceze Arvind Krishnamurthy Tianqi Chen Baris Kasikci MQ 28 132 0 29 Oct 2023
ZeroQuant-HERO: Hardware-Enhanced Robust Optimized Post-Training Quantization Framework for W8A8 Transformers Zhewei Yao Reza Yazdani Aminabadi Stephen Youn Xiaoxia Wu Elton Zheng Yuxiong He MQ 19 1 0 26 Oct 2023
Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time Zichang Liu Jue Wang Tri Dao Dinesh Manocha Binhang Yuan ... Anshumali Shrivastava Ce Zhang Yuandong Tian Christopher Ré Beidi Chen BDL 22 191 0 26 Oct 2023
QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models Elias Frantar Dan Alistarh MQ MoE 29 24 0 25 Oct 2023
Zero-Shot Sharpness-Aware Quantization for Pre-trained Language Models Miaoxi Zhu Qihuang Zhong Li Shen Liang Ding Juhua Liu Bo Du Dacheng Tao MQ VLM 29 1 0 20 Oct 2023
Matrix Compression via Randomized Low Rank and Low Precision Factorization R. Saha Varun Srivastava Mert Pilanci 26 19 0 17 Oct 2023
TEQ: Trainable Equivalent Transformation for Quantization of LLMs Wenhua Cheng Yiyang Cai Kaokao Lv Haihao Shen MQ 30 7 0 17 Oct 2023
NASH: A Simple Unified Framework of Structured Pruning for Accelerating Encoder-Decoder Language Models Jongwoo Ko Seungjoon Park Yujin Kim Sumyeong Ahn Du-Seong Chang Euijai Ahn SeYoung Yun 16 4 0 16 Oct 2023
One-Shot Sensitivity-Aware Mixed Sparsity Pruning for Large Language Models Hang Shao Bei Liu Bo Xiao Ke Zeng Guanglu Wan Yanmin Qian 44 17 0 14 Oct 2023
Dynamic Sparse No Training: Training-Free Fine-tuning for Sparse LLMs Yuxin Zhang Lirui Zhao Mingbao Lin Yunyun Sun Yiwu Yao Xingjia Han Jared Tanner Shiwei Liu Rongrong Ji SyDa 42 40 0 13 Oct 2023
QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models Jing Liu Ruihao Gong Xiuying Wei Zhiwei Dong Jianfei Cai Bohan Zhuang MQ 28 51 0 12 Oct 2023
Sparse Fine-tuning for Inference Acceleration of Large Language Models Eldar Kurtic Denis Kuznedelev Elias Frantar Michael Goin Dan Alistarh 35 8 0 10 Oct 2023
Revisiting Block-based Quantisation: What is Important for Sub-8-bit LLM Inference? Cheng Zhang Jianyi Cheng Ilia Shumailov George A. Constantinides Yiren Zhao MQ 21 9 0 08 Oct 2023
Compresso: Structured Pruning with Collaborative Prompting Learns Compact Large Language Models Song Guo Jiahang Xu Li Lyna Zhang Mao Yang 25 14 0 08 Oct 2023
Dual Grained Quantization: Efficient Fine-Grained Quantization for LLM Luoming Zhang Wen Fei Weijia Wu Yefei He Zhenyu Lou Hong Zhou MQ 22 5 0 07 Oct 2023
Dynamic Shuffle: An Efficient Channel Mixture Method Kaijun Gong Zhuowen Yin Yushu Li K. Guo Xiangmin Xu 28 0 0 04 Oct 2023
Do Compressed LLMs Forget Knowledge? An Experimental Study with Practical Implications Duc N. M. Hoang Minsik Cho Thomas Merth Mohammad Rastegari Zhangyang Wang KELM CLL 31 3 0 02 Oct 2023
A Comprehensive Review of Generative AI in Healthcare Yasin Shokrollahi Sahar Yarmohammadtoosky Matthew M. Nikahd Pengfei Dong Xianqi Li Linxia Gu MedIm AI4CE 27 19 0 01 Oct 2023
PB-LLM: Partially Binarized Large Language Models Yuzhang Shang Zhihang Yuan Qiang Wu Zhen Dong MQ 22 43 0 29 Sep 2023
ModuLoRA: Finetuning 2-Bit LLMs on Consumer GPUs by Integrating with Modular Quantizers Junjie Yin Jiahao Dong Yingheng Wang Christopher De Sa Volodymyr Kuleshov MQ 28 4 0 28 Sep 2023
Rethinking Channel Dimensions to Isolate Outliers for Low-bit Weight Quantization of Large Language Models Jung Hwan Heo Jeonghoon Kim Beomseok Kwon Byeongwook Kim Se Jung Kwon Dongsoo Lee MQ 43 9 0 27 Sep 2023
QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Models Yuhui Xu Lingxi Xie Xiaotao Gu Xin Chen Heng Chang Hengheng Zhang Zhensu Chen Xiaopeng Zhang Qi Tian MQ 21 89 0 26 Sep 2023
DreamLLM: Synergistic Multimodal Comprehension and Creation Runpei Dong Chunrui Han Yuang Peng Zekun Qi Zheng Ge ... Hao-Ran Wei Xiangwen Kong Xiangyu Zhang Kaisheng Ma Li Yi MLLM 39 173 0 20 Sep 2023