QuIP: 2-Bit Quantization of Large Language Models With Guarantees

25 July 2023

Papers citing "QuIP: 2-Bit Quantization of Large Language Models With Guarantees"

50 / 150 papers shown

Title
Laboratory-Scale AI: Open-Weight Models are Competitive with ChatGPT Even in Low-Resource Settings Robert Wolfe Isaac Slaughter Bin Han Bingbing Wen Yiwei Yang ... Bernease Herman E. Brown Zening Qu Nicholas Weber Bill Howe 43 4 0 27 May 2024
PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression Vladimir Malinovskii Denis Mazur Ivan Ilin Denis Kuznedelev Konstantin Burlachenko Kai Yi Dan Alistarh Peter Richtárik MQ 37 19 0 23 May 2024
SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Models Wei Huang Haotong Qin Yangdong Liu Yawei Li Xianglong Liu Luca Benini Michele Magno Xiaojuan Qi MQ 67 15 0 23 May 2024
Embedding Compression for Efficient Re-Identification Luke McDermott 30 0 0 23 May 2024
Integer Scale: A Free Lunch for Faster Fine-grained Quantization of LLMs Qingyuan Li Ran Meng Yiduo Li Bo Zhang Yifan Lu Yerui Sun Lin Ma Yuchen Xie MQ 38 0 0 23 May 2024
Mitigating Quantization Errors Due to Activation Spikes in GLU-Based LLMs Jaewoo Yang Hayun Kim Younghoon Kim 47 12 0 23 May 2024
OAC: Output-adaptive Calibration for Accurate Post-training Quantization Ali Edalati Alireza Ghaffari M. Asgharian Lu Hou Boxing Chen Vahid Partovi Nia V. Nia MQ 86 0 0 23 May 2024
HW-GPT-Bench: Hardware-Aware Architecture Benchmark for Language Models R. Sukthanker Arber Zela B. Staffler Aaron Klein Lennart Purucker Jorg K. H. Franke Frank Hutter ELM 43 3 0 16 May 2024
QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving Yujun Lin Haotian Tang Shang Yang Zhekai Zhang Guangxuan Xiao Chuang Gan Song Han 90 76 0 07 May 2024
Enabling High-Sparsity Foundational Llama Models with Efficient Pretraining and Deployment Abhinav Agarwalla Abhay Gupta Alexandre Marques Shubhra Pandit Michael Goin ... Tuan Nguyen Mahmoud Salem Dan Alistarh Sean Lie Mark Kurtz MoE SyDa 40 11 0 06 May 2024
Learning from Students: Applying t-Distributions to Explore Accurate and Efficient Formats for LLMs Jordan Dotzel Yuzong Chen Bahaa Kotb Sushma Prasad Gang Wu Sheng Li Mohamed S. Abdelfattah Zhiru Zhang 31 8 0 06 May 2024
HLSTransform: Energy-Efficient Llama 2 Inference on FPGAs Via High Level Synthesis Andy He Darren Key Mason Bulling Andrew Chang Skyler Shapiro Everett Lee 38 1 0 29 Apr 2024
A Survey on Efficient Inference for Large Language Models Zixuan Zhou Xuefei Ning Ke Hong Tianyu Fu Jiaming Xu ... Shengen Yan Guohao Dai Xiao-Ping Zhang Yuhan Dong Yu-Xiang Wang 46 83 0 22 Apr 2024
An empirical study of LLaMA3 quantization: from LLMs to MLLMs Wei Huang Xingyu Zheng Xudong Ma Haotong Qin Chengtao Lv Hong Chen Jie Luo Xiaojuan Qi Xianglong Liu Michele Magno MQ 59 38 0 22 Apr 2024
decoupleQ: Towards 2-bit Post-Training Uniform Quantization via decoupling Parameters into Integer and Floating Points Yi Guo Fanliu Kong Xiaoyang Li Hui Li Wei Chen Xiaogang Tian Jinping Cai Yang Zhang Shouda Liu MQ 26 6 0 19 Apr 2024
Multilingual Brain Surgeon: Large Language Models Can be Compressed Leaving No Language Behind Hongchuan Zeng Hongshen Xu Lu Chen Kai Yu 56 5 0 06 Apr 2024
Query Performance Prediction using Relevance Judgments Generated by Large Language Models Chuan Meng Negar Arabzadeh Arian Askari Mohammad Aliannejadi Maarten de Rijke LRM 37 11 0 01 Apr 2024
QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs Saleh Ashkboos Amirkeivan Mohtashami Maximilian L. Croci Bo Li Martin Jaggi Dan Alistarh Torsten Hoefler James Hensman MQ 36 135 0 30 Mar 2024
Jetfire: Efficient and Accurate Transformer Pretraining with INT8 Data Flow and Per-Block Quantization Haocheng Xi Yuxiang Chen Kang Zhao Kaijun Zheng Jianfei Chen Jun Zhu MQ 42 20 0 19 Mar 2024
Decoding Compressed Trust: Scrutinizing the Trustworthiness of Efficient LLMs Under Compression Junyuan Hong Jinhao Duan Chenhui Zhang Zhangheng Li Chulin Xie ... B. Kailkhura Dan Hendrycks Dawn Song Zhangyang Wang Bo-wen Li 39 24 0 18 Mar 2024
IM-Unpack: Training and Inference with Arbitrarily Low Precision Integers Zhanpeng Zeng Karthikeyan Sankaralingam Vikas Singh 58 1 0 12 Mar 2024
FrameQuant: Flexible Low-Bit Quantization for Transformers Harshavardhan Adepu Zhanpeng Zeng Li Zhang Vikas Singh MQ 42 6 0 10 Mar 2024
The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits Shuming Ma Hongyu Wang Lingxiao Ma Lei Wang Wenhui Wang Shaohan Huang Lifeng Dong Ruiping Wang Jilong Xue Furu Wei MQ 45 207 0 27 Feb 2024
LLM Inference Unveiled: Survey and Roofline Model Insights Zhihang Yuan Yuzhang Shang Yang Zhou Zhen Dong Zhe Zhou ... Yong Jae Lee Yan Yan Beidi Chen Guangyu Sun Kurt Keutzer 47 80 0 26 Feb 2024
WKVQuant: Quantizing Weight and Key/Value Cache for Large Language Models Gains More Yuxuan Yue Zhihang Yuan Haojie Duanmu Sifan Zhou Jianlong Wu Liqiang Nie MQ 32 42 0 19 Feb 2024
BitDistiller: Unleashing the Potential of Sub-4-Bit LLMs via Self-Distillation Dayou Du Yijia Zhang Shijie Cao Jiaqi Guo Ting Cao Xiaowen Chu Ningyi Xu MQ 46 30 0 16 Feb 2024
Any-Precision LLM: Low-Cost Deployment of Multiple, Different-Sized LLMs Yeonhong Park Jake Hyun SangLyul Cho Bonggeun Sim Jae W. Lee MQ 45 16 0 16 Feb 2024
BitDelta: Your Fine-Tune May Only Be Worth One Bit James Liu Guangxuan Xiao Kai Li Jason D. Lee Song Han Tri Dao Tianle Cai 33 21 0 15 Feb 2024
Efficient Language Adaptive Pre-training: Extending State-of-the-Art Large Language Models for Polish Szymon Ruciñski 36 5 0 15 Feb 2024
Model Compression and Efficient Inference for Large Language Models: A Survey Wenxiao Wang Wei Chen Yicong Luo Yongliu Long Zhengkai Lin Liye Zhang Binbin Lin Deng Cai Xiaofei He MQ 41 48 0 15 Feb 2024
Towards Next-Level Post-Training Quantization of Hyper-Scale Transformers Junhan Kim Kyungphil Park Chungman Lee Ho-Young Kim Joonyoung Kim Yongkweon Jeon MQ 22 2 0 14 Feb 2024
QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks Albert Tseng Jerry Chee Qingyao Sun Volodymyr Kuleshov Christopher De Sa MQ 128 101 0 06 Feb 2024
BiLLM: Pushing the Limit of Post-Training Quantization for LLMs Wei Huang Yangdong Liu Haotong Qin Ying Li Shiming Zhang Xianglong Liu Michele Magno Xiaojuan Qi MQ 82 69 0 06 Feb 2024
LQER: Low-Rank Quantization Error Reconstruction for LLMs Cheng Zhang Jianyi Cheng George A. Constantinides Yiren Zhao MQ 29 9 0 04 Feb 2024
ConSmax: Hardware-Friendly Alternative Softmax with Learnable Parameters Shiwei Liu Guanchen Tao Yifei Zou Derek Chow Zichen Fan Kauna Lei Bangfei Pan Dennis Sylvester Gregory Kielian Mehdi Saligane 32 7 0 31 Jan 2024
A Comprehensive Survey of Compression Algorithms for Language Models Seungcheol Park Jaehyeon Choi Sojin Lee U. Kang MQ 32 12 0 27 Jan 2024
Understanding the Potential of FPGA-Based Spatial Acceleration for Large Language Model Inference Hongzheng Chen Jiahao Zhang Yixiao Du Shaojie Xiang Zichao Yue Niansong Zhang Yaohui Cai Zhiru Zhang 58 34 0 23 Dec 2023
ZeroQuant(4+2): Redefining LLMs Quantization with a New FP6-Centric Strategy for Diverse Generative Tasks Xiaoxia Wu Haojun Xia Stephen Youn Zhen Zheng Shiyang Chen ... Reza Yazdani Aminabadi Yuxiong He Olatunji Ruwase Leon Song Zhewei Yao 71 8 0 14 Dec 2023
CBQ: Cross-Block Quantization for Large Language Models Xin Ding Xiaoyu Liu Zhijun Tu Yun-feng Zhang Wei Li ... Hanting Chen Yehui Tang Zhiwei Xiong Baoqun Yin Yunhe Wang MQ 36 13 0 13 Dec 2023
SiDA-MoE: Sparsity-Inspired Data-Aware Serving for Efficient and Scalable Large Mixture-of-Experts Models Zhixu Du Shiyu Li Yuhao Wu Xiangyu Jiang Jingwei Sun Qilin Zheng Yongkai Wu Ang Li Hai Helen Li Yiran Chen MoE 37 12 0 29 Oct 2023
QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models Elias Frantar Dan Alistarh MQ MoE 29 24 0 25 Oct 2023
BitNet: Scaling 1-bit Transformers for Large Language Models Hongyu Wang Shuming Ma Li Dong Shaohan Huang Huaijie Wang Lingxiao Ma Fan Yang Ruiping Wang Yi Wu Furu Wei MQ 34 97 0 17 Oct 2023
QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models Jing Liu Ruihao Gong Xiuying Wei Zhiwei Dong Jianfei Cai Bohan Zhuang MQ 28 51 0 12 Oct 2023
Sparse Fine-tuning for Inference Acceleration of Large Language Models Eldar Kurtic Denis Kuznedelev Elias Frantar Michael Goin Dan Alistarh 35 8 0 10 Oct 2023
ReLU Strikes Back: Exploiting Activation Sparsity in Large Language Models Iman Mirzadeh Keivan Alizadeh-Vahid Sachin Mehta C. C. D. Mundo Oncel Tuzel Golnoosh Samei Mohammad Rastegari Mehrdad Farajtabar 126 60 0 06 Oct 2023
PB-LLM: Partially Binarized Large Language Models Yuzhang Shang Zhihang Yuan Qiang Wu Zhen Dong MQ 28 43 0 29 Sep 2023
LORD: Low Rank Decomposition Of Monolingual Code LLMs For One-Shot Compression Ayush Kaushal Tejas Vaidhya Irina Rish 60 15 0 25 Sep 2023
OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models Wenqi Shao Yonghong Tian Zhaoyang Zhang Peng Xu Lirui Zhao Zhiqiang Li Kaipeng Zhang Peng Gao Yu Qiao Ping Luo MQ 26 176 0 25 Aug 2023
A Survey on Model Compression for Large Language Models Xunyu Zhu Jian Li Yong Liu Can Ma Weiping Wang 36 193 0 15 Aug 2023
SqueezeLLM: Dense-and-Sparse Quantization Sehoon Kim Coleman Hooper A. Gholami Zhen Dong Xiuyu Li Sheng Shen Michael W. Mahoney Kurt Keutzer MQ 29 167 0 13 Jun 2023