MiniViT: Compressing Vision Transformers with Weight Multiplexing

MiniViT: Compressing Vision Transformers with Weight Multiplexing

14 April 2022

Lu Yuan

ArXiv (abs)PDF HTML

Papers citing "MiniViT: Compressing Vision Transformers with Weight Multiplexing"

18 / 68 papers shown

Title
FacT: Factor-Tuning for Lightweight Adaptation on Vision Transformer Shibo Jie Zhi-Hong Deng 82 137 0 06 Dec 2022
HEAT: Hardware-Efficient Automatic Tensor Decomposition for Transformer Compression Jiaqi Gu Ben Keller Jean Kossaifi Anima Anandkumar Brucek Khailany David Z. Pan ViT 66 9 0 30 Nov 2022
Dynamic Feature Pruning and Consolidation for Occluded Person Re-Identification Yuteng Ye Hang Zhou Jiale Cai Chenxing Gao Youjia Zhang Junle Wang Qiang Hu Junqing Yu Wei Yang 71 6 0 27 Nov 2022
Is Smaller Always Faster? Tradeoffs in Compressing Self-Supervised Speech Transformers Tzu-Quan Lin Tsung-Huan Yang Chun-Yao Chang Kuang-Ming Chen Tzu-hsun Feng Hung-yi Lee Hao Tang 84 6 0 17 Nov 2022
KD-DETR: Knowledge Distillation for Detection Transformer with Consistent Distillation Points Sampling Yu Wang Xin Li Shengzhao Wen Fu-En Yang Wanping Zhang Gang Zhang Haocheng Feng Junyu Han 110 5 0 15 Nov 2022
BiViT: Extremely Compressed Binary Vision Transformer Yefei He Zhenyu Lou Luoming Zhang Jing Liu Weijia Wu Hong Zhou Bohan Zhuang ViT MQ 71 28 0 14 Nov 2022
A Generic Shared Attention Mechanism for Various Backbone Neural Networks Zhongzhan Huang Senwei Liang Mingfu Liang Liang Lin 111 6 0 27 Oct 2022
Deepening Neural Networks Implicitly and Locally via Recurrent Attention Strategy Shan Zhong Wushao Wen Jinghui Qin Zhongzhan Huang 57 0 0 27 Oct 2022
MobileViTv3: Mobile-Friendly Vision Transformer with Simple and Effective Fusion of Local, Global and Input Features S. Wadekar Abhishek Chaurasia ViT 157 93 0 30 Sep 2022
Attacking Compressed Vision Transformers Swapnil Parekh Devansh Shah Pratyush Shukla AAML 48 1 0 28 Sep 2022
Lightweight Transformers for Human Activity Recognition on Mobile Devices Sannara Ek François Portet P. Lalanda 83 32 0 22 Sep 2022
ViTKD: Practical Guidelines for ViT feature knowledge distillation Zhendong Yang Zhe Li Ailing Zeng Zexian Li Chun Yuan Yu Li 145 42 0 06 Sep 2022
Efficient High-Resolution Deep Learning: A Survey Arian Bakhtiarnia Qi Zhang Alexandros Iosifidis MedIm 158 21 0 26 Jul 2022
TinyViT: Fast Pretraining Distillation for Small Vision Transformers Kan Wu Jinnian Zhang Houwen Peng Mengchen Liu Bin Xiao Jianlong Fu Lu Yuan ViT 79 267 0 21 Jul 2022
I-ViT: Integer-only Quantization for Efficient Vision Transformer Inference Zhikai Li Qingyi Gu MQ 139 106 0 04 Jul 2022
Vision Transformers in 2022: An Update on Tiny ImageNet Ethan Huynh ViT 86 11 0 21 May 2022
Multi-Tailed Vision Transformer for Efficient Inference Yunke Wang Bo Du Wenyuan Wang Chang Xu ViT 347 6 0 03 Mar 2022
Xception: Deep Learning with Depthwise Separable Convolutions François Chollet MDE BDL PINN 1.6K 14,678 0 07 Oct 2016

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.