v1v2 (latest)

QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantization

11 March 2022

Xiuying Wei

Ruihao Gong

Yuhang Li

Xianglong Liu

F. Yu

VLM

ArXiv (abs)PDF HTML Github (122★)

Papers citing "QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantization"

48 / 48 papers shown

Title
NQKV: A KV Cache Quantization Scheme Based on Normal Distribution Characteristics Zhihang Cai Xingjun Zhang Zhendong Tan Zheng Wei MQ 193 0 0 22 May 2025
Pack-PTQ: Advancing Post-training Quantization of Neural Networks by Pack-wise Reconstruction Changjun Li Runqing Jiang Zhuo Song Pengpeng Yu Ye Zhang Yulan Guo MQ 131 0 0 01 May 2025
Post-Training Quantization for Diffusion Transformer via Hierarchical Timestep Grouping Ning Ding Jing Han Yuchuan Tian Chao Xu Kai Han Yehui Tang MQ 141 0 0 10 Mar 2025
Semantics Prompting Data-Free Quantization for Low-Bit Vision Transformers Mingliang Xu Yuyao Zhou Yuxin Zhang Shen Li Yong Li Chia-Wen Lin Zhanpeng Zeng Rongrong Ji MQ 314 0 0 31 Dec 2024
PTQ4VM: Post-Training Quantization for Visual Mamba Jun-gyu Jin Changhun Lee Seonggon Kim Eunhyeok Park MQ Mamba 115 2 0 29 Dec 2024
Exploring the Robustness and Transferability of Patch-Based Adversarial Attacks in Quantized Neural Networks Amira Guesmi B. Ouni Mohamed Bennai AAML 140 0 0 22 Nov 2024
Data Generation for Hardware-Friendly Post-Training Quantization Lior Dikstein Ariel Lapid Arnon Netzer H. Habi MQ 466 0 0 29 Oct 2024
IntLoRA: Integral Low-rank Adaptation of Quantized Diffusion Models Hang Guo Yawei Li Tao Dai Shu-Tao Xia Luca Benini MQ 96 2 0 29 Oct 2024
Temporal Feature Matters: A Framework for Diffusion Model Quantization Yushi Huang Ruihao Gong Xianglong Liu Jing Liu Yuhang Li Jiwen Lu Dacheng Tao DiffM MQ 96 0 0 28 Jul 2024
LRQ: Optimizing Post-Training Quantization for Large Language Models by Learning Low-Rank Weight-Scaling Matrices Jung Hyun Lee Jeonghoon Kim J. Yang S. Kwon Eunho Yang Kang Min Yoo Dongsoo Lee MQ 108 3 0 16 Jul 2024
Asymptotic Unbiased Sample Sampling to Speed Up Sharpness-Aware Minimization Jiaxin Deng Junbiao Pang Baochang Zhang 119 1 0 12 Jun 2024
LCQ: Low-Rank Codebook based Quantization for Large Language Models Wen-Pu Cai Wu-Jun Li Wu-Jun Li MQ 105 0 0 31 May 2024
MQBench: Towards Reproducible and Deployable Model Quantization Benchmark Yuhang Li Mingzhu Shen Jian Ma Yan Ren Mingxin Zhao Qi Zhang Ruihao Gong F. Yu Junjie Yan MQ 79 50 0 05 Nov 2021
Is In-Domain Data Really Needed? A Pilot Study on Cross-Domain Calibration for Network Quantization Haichao Yu Linjie Yang Humphrey Shi OOD MQ 46 6 0 16 May 2021
BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction Yuhang Li Ruihao Gong Xu Tan Yang Yang Peng Hu Qi Zhang F. Yu Wei Wang Shi Gu MQ 153 444 0 10 Feb 2021
BinaryBERT: Pushing the Limit of BERT Quantization Haoli Bai Wei Zhang Lu Hou Lifeng Shang Jing Jin Xin Jiang Qun Liu Michael Lyu Irwin King MQ 217 227 0 31 Dec 2020
MixMix: All You Need for Data-Free Compression Are Feature and Data Mixing Yuhang Li Feng Zhu Ruihao Gong Mingzhu Shen Xin Dong F. Yu Shaoqing Lu Shi Gu MQ 78 40 0 19 Nov 2020
Regularizing Neural Networks via Adversarial Model Perturbation Yaowei Zheng Richong Zhang Yongyi Mao AAML 86 99 0 10 Oct 2020
Once Quantization-Aware Training: High Performance Extremely Low-bit Architecture Search Mingzhu Shen Feng Liang Ruihao Gong Yuhang Li Chuming Li Chen Lin F. Yu Junjie Yan Wanli Ouyang MQ 66 39 0 09 Oct 2020
Sharpness-Aware Minimization for Efficiently Improving Generalization Pierre Foret Ariel Kleiner H. Mobahi Behnam Neyshabur AAML 199 1,359 0 03 Oct 2020
Up or Down? Adaptive Rounding for Post-Training Quantization Markus Nagel Rana Ali Amjad M. V. Baalen Christos Louizos Tijmen Blankevoort MQ 95 588 0 22 Apr 2020
Training with Quantization Noise for Extreme Model Compression Angela Fan Pierre Stock Benjamin Graham Edouard Grave Remi Gribonval Hervé Jégou Armand Joulin MQ 102 246 0 15 Apr 2020
Designing Network Design Spaces Ilija Radosavovic Raj Prateek Kosaraju Ross B. Girshick Kaiming He Piotr Dollár GNN 107 1,697 0 30 Mar 2020
ZeroQ: A Novel Zero Shot Quantization Framework Yaohui Cai Z. Yao Zhen Dong A. Gholami Michael W. Mahoney Kurt Keutzer MQ 101 399 0 01 Jan 2020
Fantastic Generalization Measures and Where to Find Them Yiding Jiang Behnam Neyshabur H. Mobahi Dilip Krishnan Samy Bengio AI4CE 145 611 0 04 Dec 2019
PyTorch: An Imperative Style, High-Performance Deep Learning Library Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury ... Sasank Chilamkurthy Benoit Steiner Lu Fang Junjie Bai Soumith Chintala ODL 556 42,639 0 03 Dec 2019
Loss Aware Post-training Quantization Yury Nahshan Brian Chmiel Chaim Baskin Evgenii Zheltonozhskii Ron Banner A. Bronstein A. Mendelson MQ 93 166 0 17 Nov 2019
HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks Zhen Dong Z. Yao Yaohui Cai Daiyaan Arfeen A. Gholami Michael W. Mahoney Kurt Keutzer MQ 95 282 0 10 Nov 2019
Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networks Yuhang Li Xin Dong Wei Wang MQ 66 259 0 28 Sep 2019
Forward and Backward Information Retention for Accurate Binary Neural Networks Haotong Qin Ruihao Gong Xianglong Liu Mingzhu Shen Ziran Wei F. Yu Jingkuan Song MQ 180 332 0 24 Sep 2019
Data-Free Quantization Through Weight Equalization and Bias Correction Markus Nagel M. V. Baalen Tijmen Blankevoort Max Welling MQ 75 515 0 11 Jun 2019
SWALP : Stochastic Weight Averaging in Low-Precision Training Guandao Yang Tianyi Zhang Polina Kirichenko Junwen Bai A. Wilson Christopher De Sa 68 97 0 26 Apr 2019
Learned Step Size Quantization S. K. Esser J. McKinstry Deepika Bablani R. Appuswamy D. Modha MQ 75 810 0 21 Feb 2019
Low-bit Quantization of Neural Networks for Efficient Inference Yoni Choukroun Eli Kravchik Fan Yang P. Kisilev MQ 82 364 0 18 Feb 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 95,229 0 11 Oct 2018
MnasNet: Platform-Aware Neural Architecture Search for Mobile Mingxing Tan Bo Chen Ruoming Pang Vijay Vasudevan Mark Sandler Andrew G. Howard Quoc V. Le MQ 128 3,018 0 31 Jul 2018
Averaging Weights Leads to Wider Optima and Better Generalization Pavel Izmailov Dmitrii Podoprikhin T. Garipov Dmitry Vetrov A. Wilson FedML MoMe 143 1,672 0 14 Mar 2018
MobileNetV2: Inverted Residuals and Linear Bottlenecks Mark Sandler Andrew G. Howard Menglong Zhu A. Zhmoginov Liang-Chieh Chen 213 19,353 0 13 Jan 2018
Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference Benoit Jacob S. Kligys Bo Chen Menglong Zhu Matthew Tang Andrew G. Howard Hartwig Adam Dmitry Kalenichenko MQ 167 3,143 0 15 Dec 2017
Exploring Generalization in Deep Learning Behnam Neyshabur Srinadh Bhojanapalli David A. McAllester Nathan Srebro FAtt 162 1,259 0 27 Jun 2017
Sharp Minima Can Generalize For Deep Nets Laurent Dinh Razvan Pascanu Samy Bengio Yoshua Bengio ODL 138 774 0 15 Mar 2017
Neural Architecture Search with Reinforcement Learning Barret Zoph Quoc V. Le 484 5,385 0 05 Nov 2016
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 433 2,946 0 15 Sep 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.3K 194,510 0 10 Dec 2015
Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding Song Han Huizi Mao W. Dally 3DGS 263 8,862 0 01 Oct 2015
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks Shaoqing Ren Kaiming He Ross B. Girshick Jian Sun AIMat ObjD 533 62,409 0 04 Jun 2015
Distilling the Knowledge in a Neural Network Geoffrey E. Hinton Oriol Vinyals J. Dean FedML 367 19,745 0 09 Mar 2015
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 1.7K 39,615 0 01 Sep 2014