MiniViT: Compressing Vision Transformers with Weight Multiplexing

MiniViT: Compressing Vision Transformers with Weight Multiplexing

14 April 2022

Lu Yuan

ArXiv (abs)PDF HTML

Papers citing "MiniViT: Compressing Vision Transformers with Weight Multiplexing"

18 / 68 papers shown

Title
FacT: Factor-Tuning for Lightweight Adaptation on Vision Transformer Shibo Jie Zhi-Hong Deng 82 137 0 06 Dec 2022
HEAT: Hardware-Efficient Automatic Tensor Decomposition for Transformer Compression Jiaqi Gu Ben Keller Jean Kossaifi Anima Anandkumar Brucek Khailany David Z. Pan ViT 66 9 0 30 Nov 2022
Dynamic Feature Pruning and Consolidation for Occluded Person Re-Identification Yuteng Ye Hang Zhou Jiale Cai Chenxing Gao Youjia Zhang Junle Wang Qiang Hu Junqing Yu Wei Yang 71 6 0 27 Nov 2022
Is Smaller Always Faster? Tradeoffs in Compressing Self-Supervised Speech Transformers Tzu-Quan Lin Tsung-Huan Yang Chun-Yao Chang Kuang-Ming Chen Tzu-hsun Feng Hung-yi Lee Hao Tang 84 6 0 17 Nov 2022
KD-DETR: Knowledge Distillation for Detection Transformer with Consistent Distillation Points Sampling Yu Wang Xin Li Shengzhao Wen Fu-En Yang Wanping Zhang Gang Zhang Haocheng Feng Junyu Han 110 5 0 15 Nov 2022
BiViT: Extremely Compressed Binary Vision Transformer Yefei He Zhenyu Lou Luoming Zhang Jing Liu Weijia Wu Hong Zhou Bohan Zhuang ViT MQ 71 28 0 14 Nov 2022
A Generic Shared Attention Mechanism for Various Backbone Neural Networks Zhongzhan Huang Senwei Liang Mingfu Liang Liang Lin 111 6 0 27 Oct 2022
Deepening Neural Networks Implicitly and Locally via Recurrent Attention Strategy Shan Zhong Wushao Wen Jinghui Qin Zhongzhan Huang 57 0 0 27 Oct 2022
MobileViTv3: Mobile-Friendly Vision Transformer with Simple and Effective Fusion of Local, Global and Input Features S. Wadekar Abhishek Chaurasia ViT 157 93 0 30 Sep 2022
Attacking Compressed Vision Transformers Swapnil Parekh Devansh Shah Pratyush Shukla AAML 48 1 0 28 Sep 2022
Lightweight Transformers for Human Activity Recognition on Mobile Devices Sannara Ek François Portet P. Lalanda 83 32 0 22 Sep 2022
ViTKD: Practical Guidelines for ViT feature knowledge distillation Zhendong Yang Zhe Li Ailing Zeng Zexian Li Chun Yuan Yu Li 145 42 0 06 Sep 2022
Efficient High-Resolution Deep Learning: A Survey Arian Bakhtiarnia Qi Zhang Alexandros Iosifidis MedIm 158 21 0 26 Jul 2022
TinyViT: Fast Pretraining Distillation for Small Vision Transformers Kan Wu Jinnian Zhang Houwen Peng Mengchen Liu Bin Xiao Jianlong Fu Lu Yuan ViT 79 267 0 21 Jul 2022
I-ViT: Integer-only Quantization for Efficient Vision Transformer Inference Zhikai Li Qingyi Gu MQ 139 106 0 04 Jul 2022
Vision Transformers in 2022: An Update on Tiny ImageNet Ethan Huynh ViT 86 11 0 21 May 2022
Multi-Tailed Vision Transformer for Efficient Inference Yunke Wang Bo Du Wenyuan Wang Chang Xu ViT 347 6 0 03 Mar 2022
Xception: Deep Learning with Depthwise Separable Convolutions François Chollet MDE BDL PINN 1.6K 14,678 0 07 Oct 2016