ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers

4 June 2022

Z. Yao

Reza Yazdani Aminabadi

Yuxiong He

Papers citing "ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers"

50 / 324 papers shown

Title
PTQ4SAM: Post-Training Quantization for Segment Anything Chengtao Lv Hong Chen Jinyang Guo Yifu Ding Xianglong Liu VLM MQ 31 13 0 06 May 2024
COPAL: Continual Pruning in Large Language Generative Models Srikanth Malla Joon Hee Choi Chiho Choi VLM CLL 34 1 0 02 May 2024
BASS: Batched Attention-optimized Speculative Sampling Haifeng Qian Sujan Kumar Gonugondla Sungsoo Ha Mingyue Shang Sanjay Krishna Gouda Ramesh Nallapati Sudipta Sengupta Xiaofei Ma Anoop Deoras BDL 50 8 0 24 Apr 2024
Mélange: Cost Efficient Large Language Model Serving by Exploiting GPU Heterogeneity Tyler Griggs Xiaoxuan Liu Jiaxiang Yu Doyoung Kim Wei-Lin Chiang Alvin Cheung Ion Stoica 54 16 0 22 Apr 2024
A Survey on Efficient Inference for Large Language Models Zixuan Zhou Xuefei Ning Ke Hong Tianyu Fu Jiaming Xu ... Shengen Yan Guohao Dai Xiao-Ping Zhang Yuhan Dong Yu-Xiang Wang 46 83 0 22 Apr 2024
Parallel Decoding via Hidden Transfer for Lossless Large Language Model Acceleration Pengfei Wu Jiahao Liu Zhuocheng Gong Qifan Wang Jinpeng Li Jingang Wang Xunliang Cai Dongyan Zhao 25 1 0 18 Apr 2024
LoRAP: Transformer Sub-Layers Deserve Differentiated Structured Compression for Large Language Models Guangyan Li Yongqiang Tang Wensheng Zhang 49 5 0 15 Apr 2024
Exploring and Improving Drafts in Blockwise Parallel Decoding Taehyeon Kim A. Suresh Kishore Papineni Michael Riley Sanjiv Kumar Adrian Benton AI4TS 49 2 0 14 Apr 2024
Frame Quantization of Neural Networks Wojciech Czaja Sanghoon Na 32 1 0 11 Apr 2024
Investigating the Impact of Quantization on Adversarial Robustness Qun Li Yuan Meng Chen Tang Jiacheng Jiang Zhi Wang 22 3 0 08 Apr 2024
Multilingual Brain Surgeon: Large Language Models Can be Compressed Leaving No Language Behind Hongchuan Zeng Hongshen Xu Lu Chen Kai Yu 56 5 0 06 Apr 2024
Minimize Quantization Output Error with Bias Compensation Cheng Gong Haoshuai Zheng Mengting Hu Zheng Lin Deng-Ping Fan Yuzhi Zhang Tao Li MQ 38 2 0 02 Apr 2024
Oh! We Freeze: Improving Quantized Knowledge Distillation via Signal Propagation Analysis for Large Language Models Kartikeya Bhardwaj N. Pandey Sweta Priyadarshi Kyunggeun Lee Jun Ma Harris Teague MQ 42 2 0 26 Mar 2024
AI and Memory Wall A. Gholami Z. Yao Sehoon Kim Coleman Hooper Michael W. Mahoney Kurt Keutzer 27 143 0 21 Mar 2024
Efficient Encoder-Decoder Transformer Decoding for Decomposable Tasks Bo-Ru Lu Nikita Haduong Chien-Yu Lin Hao Cheng Noah A. Smith Mari Ostendorf AI4CE 40 0 0 19 Mar 2024
AffineQuant: Affine Transformation Quantization for Large Language Models Yuexiao Ma Huixia Li Xiawu Zheng Feng Ling Xuefeng Xiao Rui Wang Shilei Wen Rongrong Ji Rongrong Ji MQ 43 19 0 19 Mar 2024
ExeGPT: Constraint-Aware Resource Scheduling for LLM Inference Hyungjun Oh Kihong Kim Jaemin Kim Sungkyun Kim Junyeol Lee Du-Seong Chang Jiwon Seo 41 28 0 15 Mar 2024
PYRA: Parallel Yielding Re-Activation for Training-Inference Efficient Task Adaptation Yizhe Xiong Hui Chen Tianxiang Hao Zijia Lin Jungong Han Yuesong Zhang Guoxin Wang Yongjun Bao Guiguang Ding 51 17 0 14 Mar 2024
Bifurcated Attention: Accelerating Massively Parallel Decoding with Shared Prefixes in LLMs Ben Athiwaratkun Sujan Kumar Gonugondla Sanjay Krishna Gouda Haifeng Qian Hantian Ding ... Liangfu Chen Parminder Bhatia Ramesh Nallapati Sudipta Sengupta Bing Xiang 59 4 0 13 Mar 2024
FrameQuant: Flexible Low-Bit Quantization for Transformers Harshavardhan Adepu Zhanpeng Zeng Li Zhang Vikas Singh MQ 45 6 0 10 Mar 2024
GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLM Hao Kang Qingru Zhang Souvik Kundu Geonhwa Jeong Zaoxing Liu Tushar Krishna Tuo Zhao MQ 43 81 0 08 Mar 2024
QAQ: Quality Adaptive Quantization for LLM KV Cache Shichen Dong Wenfang Cheng Jiayu Qin Wei Wang MQ 49 34 0 07 Mar 2024
EasyQuant: An Efficient Data-free Quantization Algorithm for LLMs Hanlin Tang Yifu Sun Decheng Wu Kai Liu Jianchen Zhu Zhanhui Kang MQ 28 10 0 05 Mar 2024
On the Compressibility of Quantized Large Language Models Yu Mao Weilan Wang Hongchao Du Nan Guan Chun Jason Xue MQ 36 6 0 03 Mar 2024
OSSCAR: One-Shot Structured Pruning in Vision and Language Models with Combinatorial Optimization Xiang Meng Shibal Ibrahim Kayhan Behdin Hussein Hazimeh Natalia Ponomareva Rahul Mazumder VLM 49 5 0 02 Mar 2024
LLM-PQ: Serving LLM on Heterogeneous Clusters with Phase-Aware Partition and Adaptive Quantization Juntao Zhao Borui Wan Size Zheng Yanghua Peng Chuan Wu MQ 41 13 0 02 Mar 2024
Evaluating Quantized Large Language Models Shiyao Li Xuefei Ning Luning Wang Tengxuan Liu Xiangsheng Shi Shengen Yan Guohao Dai Huazhong Yang Yu-Xiang Wang MQ 45 45 0 28 Feb 2024
SparseLLM: Towards Global Pruning for Pre-trained Language Models Guangji Bai Yijiang Li Chen Ling Kibaek Kim Liang Zhao 30 6 0 28 Feb 2024
On the Challenges and Opportunities in Generative AI Laura Manduchi Kushagra Pandey Robert Bamler Ryan Cotterell Sina Daubener ... F. Wenzel Frank Wood Stephan Mandt Vincent Fortuin Vincent Fortuin 56 17 0 28 Feb 2024
Think Big, Generate Quick: LLM-to-SLM for Fast Autoregressive Decoding Benjamin Bergner Andrii Skliar Amelie Royer Tijmen Blankevoort Yuki Markus Asano B. Bejnordi 58 5 0 26 Feb 2024
A Comprehensive Evaluation of Quantization Strategies for Large Language Models Renren Jin Jiangcun Du Wuwei Huang Wei Liu Jian Luan Bin Wang Deyi Xiong MQ 32 31 0 26 Feb 2024
LLM Inference Unveiled: Survey and Roofline Model Insights Zhihang Yuan Yuzhang Shang Yang Zhou Zhen Dong Zhe Zhou ... Yong Jae Lee Yan Yan Beidi Chen Guangyu Sun Kurt Keutzer 50 80 0 26 Feb 2024
Data-free Weight Compress and Denoise for Large Language Models Runyu Peng Yunhua Zhou Qipeng Guo Yang Gao Hang Yan Xipeng Qiu Dahua Lin 41 1 0 26 Feb 2024
MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases Zechun Liu Changsheng Zhao Forrest N. Iandola Chen Lai Yuandong Tian ... Ernie Chang Yangyang Shi Raghuraman Krishnamoorthi Liangzhen Lai Vikas Chandra ALM 43 77 0 22 Feb 2024
A Survey on Knowledge Distillation of Large Language Models Xiaohan Xu Ming Li Chongyang Tao Tao Shen Reynold Cheng Jinyang Li Can Xu Dacheng Tao Dinesh Manocha KELM VLM 44 102 0 20 Feb 2024
WKVQuant: Quantizing Weight and Key/Value Cache for Large Language Models Gains More Yuxuan Yue Zhihang Yuan Haojie Duanmu Sifan Zhou Jianlong Wu Liqiang Nie MQ 34 42 0 19 Feb 2024
CliqueParcel: An Approach For Batching LLM Prompts That Jointly Optimizes Efficiency And Faithfulness Jiayi Liu Tinghan Yang Jennifer Neville 26 10 0 17 Feb 2024
LaCo: Large Language Model Pruning via Layer Collapse Yifei Yang Zouying Cao Hai Zhao 21 52 0 17 Feb 2024
Speculative Streaming: Fast LLM Inference without Auxiliary Models Nikhil Bhendawade Irina Belousova Qichen Fu Henry Mason Mohammad Rastegari Mahyar Najibi LRM 34 28 0 16 Feb 2024
EdgeQAT: Entropy and Distribution Guided Quantization-Aware Training for the Acceleration of Lightweight LLMs on the Edge Xuan Shen Zhenglun Kong Changdi Yang Zhaoyang Han Lei Lu ... Zhihao Shu Wei Niu Miriam Leeser Pu Zhao Yanzhi Wang MQ 51 18 0 16 Feb 2024
Model Compression and Efficient Inference for Large Language Models: A Survey Wenxiao Wang Wei Chen Yicong Luo Yongliu Long Zhengkai Lin Liye Zhang Binbin Lin Deng Cai Xiaofei He MQ 41 48 0 15 Feb 2024
Towards Meta-Pruning via Optimal Transport Alexander Theus Olin Geimer Friedrich Wicke Thomas Hofmann Sotiris Anagnostidis Sidak Pal Singh MoMe 24 3 0 12 Feb 2024
RepQuant: Towards Accurate Post-Training Quantization of Large Transformer Models via Scale Reparameterization Zhikai Li Xuewen Liu Jing Zhang Qingyi Gu MQ 45 7 0 08 Feb 2024
BiLLM: Pushing the Limit of Post-Training Quantization for LLMs Wei Huang Yangdong Liu Haotong Qin Ying Li Shiming Zhang Xianglong Liu Michele Magno Xiaojuan Qi MQ 82 69 0 06 Feb 2024
ReLU $^2$ Wins: Discovering Efficient Activation Functions for Sparse LLMs Zhengyan Zhang Yixin Song Guanghui Yu Xu Han Yankai Lin Chaojun Xiao Chenyang Song Zhiyuan Liu Zeyu Mi Maosong Sun 22 31 0 06 Feb 2024
Progress and Opportunities of Foundation Models in Bioinformatics Qing Li Zhihang Hu Yixuan Wang Lei Li Yimin Fan Irwin King Le Song Yu Li AI4CE 43 9 0 06 Feb 2024
A Survey on Transformer Compression Yehui Tang Yunhe Wang Jianyuan Guo Zhijun Tu Kai Han Hailin Hu Dacheng Tao 37 28 0 05 Feb 2024
A Comprehensive Survey of Compression Algorithms for Language Models Seungcheol Park Jaehyeon Choi Sojin Lee U. Kang MQ 32 12 0 27 Jan 2024
Inference without Interference: Disaggregate LLM Inference for Mixed Downstream Workloads Cunchen Hu Heyang Huang Liangliang Xu Xusheng Chen Jiang Xu ... Chenxi Wang Sa Wang Yungang Bao Ninghui Sun Yizhou Shan DRL 35 62 0 20 Jan 2024
Inferflow: an Efficient and Highly Configurable Inference Engine for Large Language Models Shuming Shi Enbo Zhao Deng Cai Leyang Cui Xinting Huang Huayang Li 28 3 0 16 Jan 2024