ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers

4 June 2022

Z. Yao

Reza Yazdani Aminabadi

Yuxiong He

Papers citing "ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers"

50 / 324 papers shown

Title
Accelerating Large Language Model Inference with Self-Supervised Early Exits Florian Valade LRM 44 1 0 30 Jul 2024
Accurate and Efficient Fine-Tuning of Quantized Large Language Models Through Optimal Balance Ao Shen Qiang Wang Zhiquan Lai Xionglve Li Dongsheng Li ALM MQ 32 1 0 24 Jul 2024
Compensate Quantization Errors+: Quantized Models Are Inquisitive Learners Yifei Gao Jie Ou Lei Wang Fanhua Shang Jaji Wu MQ 49 0 0 22 Jul 2024
LRQ: Optimizing Post-Training Quantization for Large Language Models by Learning Low-Rank Weight-Scaling Matrices Jung Hyun Lee Jeonghoon Kim J. Yang S. Kwon Eunho Yang Kang Min Yoo Dongsoo Lee MQ 36 2 0 16 Jul 2024
Composable Interventions for Language Models Arinbjorn Kolbeinsson Kyle O'Brien Tianjin Huang Shanghua Gao Shiwei Liu ... Anurag J. Vaidya Faisal Mahmood Marinka Zitnik Tianlong Chen Thomas Hartvigsen KELM MU 89 5 0 09 Jul 2024
GPTQT: Quantize Large Language Models Twice to Push the Efficiency Yipin Guo Yilin Lang Qinyuan Ren MQ 24 1 0 03 Jul 2024
Let the Code LLM Edit Itself When You Edit the Code Zhenyu He Jun Zhang Shengjie Luo Jingjing Xu Z. Zhang Di He KELM 36 0 0 03 Jul 2024
LLMEasyQuant: Scalable Quantization for Parallel and Distributed LLM Inference Dong Liu Meng Jiang MQ 36 12 0 28 Jun 2024
OutlierTune: Efficient Channel-Wise Quantization for Large Language Models Jinguang Wang Yuexi Yin Haifeng Sun Qi Qi Jingyu Wang Zirui Zhuang Tingting Yang Jianxin Liao 43 2 0 27 Jun 2024
MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Pool Cunchen Hu Heyang Huang Junhao Hu Jiang Xu Xusheng Chen ... Chenxi Wang Sa Wang Yungang Bao Ninghui Sun Yizhou Shan LLMAG 50 23 0 25 Jun 2024
Layer-Wise Quantization: A Pragmatic and Effective Method for Quantizing LLMs Beyond Integer Bit-Levels Razvan-Gabriel Dumitru Vikas Yadav Rishabh Maheshwary Paul-Ioan Clotan Sathwik Tejaswi Madhusudhan Mihai Surdeanu MQ 43 2 0 25 Jun 2024
Compensate Quantization Errors: Make Weights Hierarchical to Compensate Each Other Yifei Gao Jie Ou Lei Wang Yuting Xiao Zhiyuan Xiang Ruiting Dai Jun Cheng MQ 36 3 0 24 Jun 2024
BoA: Attention-aware Post-training Quantization without Backpropagation Junhan Kim Ho-Young Kim Eulrang Cho Chungman Lee Joonyoung Kim Yongkweon Jeon MQ 38 0 0 19 Jun 2024
Prefixing Attention Sinks can Mitigate Activation Outliers for Large Language Model Quantization Seungwoo Son Wonpyo Park Woohyun Han Kyuyeun Kim Jaeho Lee MQ 34 10 0 17 Jun 2024
Endor: Hardware-Friendly Sparse Format for Offloaded LLM Inference Donghyeon Joo Ramyad Hadidi S. Feizi Bahar Asgari MQ 31 0 0 17 Jun 2024
Outlier Reduction with Gated Attention for Improved Post-training Quantization in Large Sequence-to-sequence Speech Foundation Models Dominik Wagner Ilja Baumann Korbinian Riedhammer Tobias Bocklet MQ 32 1 0 16 Jun 2024
Tender: Accelerating Large Language Models via Tensor Decomposition and Runtime Requantization Jungi Lee Wonbeom Lee Jaewoong Sim MQ 42 14 0 16 Jun 2024
Evaluating the Generalization Ability of Quantized LLMs: Benchmark, Analysis, and Toolbox Yijun Liu Yuan Meng Fang Wu Shenhao Peng Hang Yao Chaoyu Guan Chen Tang Xinzhu Ma Zhi Wang Wenwu Zhu MQ 58 7 0 15 Jun 2024
BlockPruner: Fine-grained Pruning for Large Language Models Longguang Zhong Fanqi Wan Ruijun Chen Xiaojun Quan Liangzhi Li 31 7 0 15 Jun 2024
AdaPTwin: Low-Cost Adaptive Compression of Product Twins in Transformers Emil Biju Anirudh Sriram Mert Pilanci 54 0 0 13 Jun 2024
ALPS: Improved Optimization for Highly Sparse One-Shot Pruning for Large Language Models Xiang Meng Kayhan Behdin Haoyue Wang Rahul Mazumder 42 3 0 12 Jun 2024
MoreauPruner: Robust Pruning of Large Language Models against Weight Perturbations Zixiao Wang Jingwei Zhang Wenqian Zhao Farzan Farnia Bei Yu AAML 37 3 0 11 Jun 2024
Low-Rank Quantization-Aware Training for LLMs Yelysei Bondarenko Riccardo Del Chiaro Markus Nagel MQ 33 10 0 10 Jun 2024
ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterization Haoran You Yipin Guo Yichao Fu Wei Zhou Huihong Shi Xiaofan Zhang Souvik Kundu Amir Yazdanbakhsh Y. Lin KELM 56 7 0 10 Jun 2024
Turbo Sparse: Achieving LLM SOTA Performance with Minimal Activated Parameters Yixin Song Haotong Xie Zhengyan Zhang Bo Wen Li Ma Zeyu Mi Haibo Chen MoE 41 21 0 10 Jun 2024
Enabling Efficient Batch Serving for LMaaS via Generation Length Prediction Ke Cheng Wen Hu Zhi Wang Peng Du Jianguo Li Sheng Zhang 42 10 0 07 Jun 2024
Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for Large Language Models Peijie Dong Lujun Li Zhenheng Tang Xiang Liu Xinglin Pan Qiang-qiang Wang Xiaowen Chu 62 23 0 05 Jun 2024
ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation Tianchen Zhao Tongcheng Fang Haofeng Huang Enshu Liu Widyadewi Soedarmadji ... Shengen Yan Huazhong Yang Xuefei Ning Xuefei Ning Yu Wang MQ VGen 112 26 0 04 Jun 2024
SUBLLM: A Novel Efficient Architecture with Token Sequence Subsampling for LLM Quandong Wang Yuxuan Yuan Xiaoyu Yang Ruike Zhang Kang Zhao Wei Liu Jian Luan Daniel Povey Bin Wang 49 0 0 03 Jun 2024
LCQ: Low-Rank Codebook based Quantization for Large Language Models Wen-Pu Cai Wu-Jun Li Wu-Jun Li MQ 46 0 0 31 May 2024
FinerCut: Finer-grained Interpretable Layer Pruning for Large Language Models Yang Zhang Yawei Li Xinpeng Wang Qianli Shen Barbara Plank Bernd Bischl Mina Rezaei Kenji Kawaguchi 60 7 0 28 May 2024
I-LLM: Efficient Integer-Only Inference for Fully-Quantized Low-Bit Large Language Models Xing Hu Yuan Cheng Dawei Yang Zhihang Yuan Jiangyong Yu Chen Xu Sifan Zhou MQ 36 7 0 28 May 2024
CLAQ: Pushing the Limits of Low-Bit Post-Training Quantization for LLMs Haoyu Wang Bei Liu Hang Shao Bo Xiao Ke Zeng Guanglu Wan Yanmin Qian MQ 29 0 0 27 May 2024
TURNIP: A "Nondeterministic" GPU Runtime with CPU RAM Offload Zhimin Ding Jiawen Yao Brianna Barrow Tania Lorido-Botran Christopher M. Jermaine Yu-Shuen Tang Jiehui Li Xinyu Yao Sleem Mahmoud Abdelghafar Daniel Bourgeois 46 1 0 25 May 2024
FastQuery: Communication-efficient Embedding Table Query for Private LLM Inference Chenqi Lin Tianshi Xu Zebin Yang Runsheng Wang Ru Huang Meng Li 33 0 0 25 May 2024
PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression Vladimir Malinovskii Denis Mazur Ivan Ilin Denis Kuznedelev Konstantin Burlachenko Kai Yi Dan Alistarh Peter Richtárik MQ 37 19 0 23 May 2024
SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Models Wei Huang Haotong Qin Yangdong Liu Yawei Li Xianglong Liu Luca Benini Michele Magno Xiaojuan Qi MQ 67 15 0 23 May 2024
Integer Scale: A Free Lunch for Faster Fine-grained Quantization of LLMs Qingyuan Li Ran Meng Yiduo Li Bo Zhang Yifan Lu Yerui Sun Lin Ma Yuchen Xie MQ 38 0 0 23 May 2024
Mitigating Quantization Errors Due to Activation Spikes in GLU-Based LLMs Jaewoo Yang Hayun Kim Younghoon Kim 47 12 0 23 May 2024
ZipCache: Accurate and Efficient KV Cache Quantization with Salient Token Identification Yefei He Luoming Zhang Weijia Wu Jing Liu Hong Zhou Bohan Zhuang MQ 51 25 0 23 May 2024
OAC: Output-adaptive Calibration for Accurate Post-training Quantization Ali Edalati Alireza Ghaffari M. Asgharian Lu Hou Boxing Chen Vahid Partovi Nia V. Nia MQ 86 0 0 23 May 2024
eXmY: A Data Type and Technique for Arbitrary Bit Precision Quantization Aditya Agrawal Matthew Hedlund Blake A. Hechtman MQ 33 4 0 22 May 2024
QGait: Toward Accurate Quantization for Gait Recognition with Binarized Input Senmao Tian Haoyu Gao Gangyi Hong Shuyun Wang JingJie Wang Xin Yu Shunli Zhang MQ 40 1 0 22 May 2024
AdpQ: A Zero-shot Calibration Free Adaptive Post Training Quantization Method for LLMs Alireza Ghaffari Sharareh Younesian Vahid Partovi Nia Boxing Chen M. Asgharian MQ 52 0 0 22 May 2024
ReALLM: A general framework for LLM compression and fine-tuning Louis Leconte Lisa Bedin Van Minh Nguyen Eric Moulines MQ 41 0 0 21 May 2024
Characterizing the Accuracy - Efficiency Trade-off of Low-rank Decomposition in Language Models Chakshu Moar Michael Pellauer Hyoukjun Kwon 38 1 0 10 May 2024
From Algorithm to Hardware: A Survey on Efficient and Safe Deployment of Deep Neural Networks Xue Geng Zhe Wang Chunyun Chen Qing Xu Kaixin Xu ... Zhenghua Chen M. Aly Jie Lin Min-man Wu Xiaoli Li 33 1 0 09 May 2024
LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit Ruihao Gong Yang Yong Shiqiao Gu Yushi Huang Chentao Lv Yunchen Zhang Xianglong Liu Dacheng Tao MQ 42 7 0 09 May 2024
Ditto: Quantization-aware Secure Inference of Transformers upon MPC Haoqi Wu Wenjing Fang Yancheng Zheng Junming Ma Jin Tan Yinggui Wang Lei Wang MQ 53 2 0 09 May 2024
Enabling High-Sparsity Foundational Llama Models with Efficient Pretraining and Deployment Abhinav Agarwalla Abhay Gupta Alexandre Marques Shubhra Pandit Michael Goin ... Tuan Nguyen Mahmoud Salem Dan Alistarh Sean Lie Mark Kurtz MoE SyDa 40 11 0 06 May 2024