Oscillation-free Quantization for Low-bit Vision Transformers

4 February 2023

Papers citing "Oscillation-free Quantization for Low-bit Vision Transformers"

26 / 26 papers shown

Title
Les Dissonances: Cross-Tool Harvesting and Polluting in Multi-Tool Empowered LLM Agents Zichuan Li Jian Cui Xiaojing Liao Luyi Xing LLMAG 40 0 0 04 Apr 2025
SSVQ: Unleashing the Potential of Vector Quantization with Sign-Splitting Shuaiting Li Juncan Deng Chenxuan Wang Kedong Xu Rongtao Deng Hong Gu Haibin Shen Kejie Huang MQ 53 0 0 11 Mar 2025
AHCPTQ: Accurate and Hardware-Compatible Post-Training Quantization for Segment Anything Model Wenlun Zhang Shimpei Ando Kentaro Yoshioka VLM MQ 67 0 0 05 Mar 2025
Q&C: When Quantization Meets Cache in Efficient Image Generation Xin Ding X. Li Haotong Qin Zhibo Chen DiffM MQ 75 0 0 04 Mar 2025
Oscillation-Reduced MXFP4 Training for Vision Transformers Yuxiang Chen Haocheng Xi Jun Zhu Jianfei Chen MQ 56 2 0 28 Feb 2025
Mix-QViT: Mixed-Precision Vision Transformer Quantization Driven by Layer Importance and Quantization Sensitivity Navin Ranjan Andreas E. Savakis MQ 47 1 0 10 Jan 2025
Semantics Prompting Data-Free Quantization for Low-Bit Vision Transformers Yunshan Zhong Yuyao Zhou Yuxin Zhang Shen Li Yong Li Rongrong Ji Zhanpeng Zeng Rongrong Ji MQ 94 0 0 31 Dec 2024
EoRA: Training-free Compensation for Compressed LLM with Eigenspace Low-Rank Approximation Shih-yang Liu Huck Yang Nai Chit Fung Nai Chit Fung Hongxu Yin ... Jan Kautz Yu-Chun Wang Pavlo Molchanov Min-Hung Chen Min-Hung Chen MQ 31 0 0 28 Oct 2024
Gap Preserving Distillation by Building Bidirectional Mappings with A Dynamic Teacher Yong Guo Shulian Zhang Haolin Pan Jing Liu Yulun Zhang Jian Chen 35 0 0 05 Oct 2024
Token Compensator: Altering Inference Cost of Vision Transformer without Re-Tuning Shibo Jie Yehui Tang Jianyuan Guo Zhi-Hong Deng Kai Han Yunhe Wang VLM 38 2 0 13 Aug 2024
DopQ-ViT: Towards Distribution-Friendly and Outlier-Aware Post-Training Quantization for Vision Transformers Lianwei Yang Haisong Gong Qingyi Gu MQ 37 3 0 06 Aug 2024
Mixed Non-linear Quantization for Vision Transformers Gihwan Kim Jemin Lee Sihyeong Park Yongin Kwon Hyungshin Kim MQ 35 0 0 26 Jul 2024
PTQ4DiT: Post-training Quantization for Diffusion Transformers Junyi Wu Haoxuan Wang Yuzhang Shang Mubarak Shah Yan Yan MQ 33 19 0 25 May 2024
Model Quantization and Hardware Acceleration for Vision Transformers: A Comprehensive Survey Dayou Du Gu Gong Xiaowen Chu MQ 38 7 0 01 May 2024
Genetic Quantization-Aware Approximation for Non-Linear Operations in Transformers Pingcheng Dong Yonghao Tan Dong Zhang Tianwei Ni Xuejiao Liu ... Xijie Huang Huaiyu Zhu Yun Pan Fengwei An Kwang-Ting Cheng MQ 30 3 0 28 Mar 2024
Not All Attention is Needed: Parameter and Computation Efficient Transfer Learning for Multi-modal Large Language Models Qiong Wu Weihao Ye Yiyi Zhou Xiaoshuai Sun Rongrong Ji MoE 49 1 0 22 Mar 2024
A Survey on Transformer Compression Yehui Tang Yunhe Wang Jianyuan Guo Zhijun Tu Kai Han Hailin Hu Dacheng Tao 34 28 0 05 Feb 2024
OnDev-LCT: On-Device Lightweight Convolutional Transformers towards federated learning Chu Myaet Thwal Minh N. H. Nguyen Ye Lin Tun Seongjin Kim My T. Thai Choong Seon Hong 61 5 0 22 Jan 2024
I&S-ViT: An Inclusive & Stable Method for Pushing the Limit of Post-Training ViTs Quantization Yunshan Zhong Jiawei Hu Mingbao Lin Mengzhao Chen Rongrong Ji MQ 30 10 0 16 Nov 2023
Reducing the Side-Effects of Oscillations in Training of Quantized YOLO Networks Kartik Gupta Akshay Asthana MQ 24 8 0 09 Nov 2023
Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs Wenhua Cheng Weiwei Zhang Haihao Shen Yiyang Cai Xin He Kaokao Lv Yi. Liu MQ 24 21 0 11 Sep 2023
Jumping through Local Minima: Quantization in the Loss Landscape of Vision Transformers N. Frumkin Dibakar Gope Diana Marculescu MQ 41 16 0 21 Aug 2023
LLM-QAT: Data-Free Quantization Aware Training for Large Language Models Zechun Liu Barlas Oğuz Changsheng Zhao Ernie Chang Pierre Stock Yashar Mehdad Yangyang Shi Raghuraman Krishnamoorthi Vikas Chandra MQ 46 188 0 29 May 2023
DenseShift: Towards Accurate and Efficient Low-Bit Power-of-Two Quantization Xinlin Li Bangya Liu Ruizhi Yang Vanessa Courville Chao Xing V. Nia MQ 34 2 0 20 Aug 2022
I-ViT: Integer-only Quantization for Efficient Vision Transformer Inference Zhikai Li Qingyi Gu MQ 54 95 0 04 Jul 2022
Overcoming Oscillations in Quantization-Aware Training Markus Nagel Marios Fournarakis Yelysei Bondarenko Tijmen Blankevoort MQ 111 101 0 21 Mar 2022