MiniViT: Compressing Vision Transformers with Weight Multiplexing

14 April 2022

Lu Yuan

Papers citing "MiniViT: Compressing Vision Transformers with Weight Multiplexing"

50 / 68 papers shown

Title
General Compression Framework for Efficient Transformer Object Tracking Lingyi Hong Jinglun Li Xinyu Zhou Shilin Yan Pinxue Guo ... Runze Li Xingdong Sheng Wei Zhang Hong Lu Wenqiang Zhang ViT 97 1 0 01 Jul 2025
LightNobel: Improving Sequence Length Limitation in Protein Structure Prediction Model via Adaptive Activation Quantization Seunghee Han S. Choi Joo-Young Kim 61 0 0 09 May 2025
Pyramid-based Mamba Multi-class Unsupervised Anomaly Detection Nasar Iqbal Niki Martinel Mamba 79 1 0 04 Apr 2025
KernelDNA: Dynamic Kernel Sharing via Decoupled Naive Adapters Haiduo Huang Yadong Zhang Pengju Ren 119 0 0 30 Mar 2025
Similarity-Guided Layer-Adaptive Vision Transformer for UAV Tracking Chaocan Xue Bineng Zhong Qihua Liang Yaozong Zheng Ning Li Yuanliang Xue Shuxiang Song 75 0 0 09 Mar 2025
Not Every Patch is Needed: Towards a More Efficient and Effective Backbone for Video-based Person Re-identification Lanyun Zhu Tianrun Chen Deyi Ji Jieping Ye Jing Liu 156 2 0 28 Jan 2025
Mix-QViT: Mixed-Precision Vision Transformer Quantization Driven by Layer Importance and Quantization Sensitivity Navin Ranjan Andreas E. Savakis MQ 88 1 0 10 Jan 2025
Semantic Alignment and Reinforcement for Data-Free Quantization of Vision Transformers Mingliang Xu Yuyao Zhou Yuxin Zhang Shen Li Shen Li Yong Li Zhanpeng Zeng Rongrong Ji MQ 333 0 0 21 Dec 2024
Gap Preserving Distillation by Building Bidirectional Mappings with A Dynamic Teacher Yong Guo Shulian Zhang Haolin Pan Jing Liu Yulun Zhang Jian Chen 87 0 0 05 Oct 2024
FINE: Factorizing Knowledge for Initialization of Variable-sized Diffusion Models Yucheng Xie Fu Feng Ruixiao Shi Jing Wang Xin Geng AI4CE 79 3 0 28 Sep 2024
Lessons and Insights from a Unifying Study of Parameter-Efficient Fine-Tuning (PEFT) in Visual Recognition Zheda Mai Ping Zhang Cheng-Hao Tu Hong-You Chen Li Zhang Wei-Lun Chao 52 1 0 24 Sep 2024
TReX- Reusing Vision Transformer's Attention for Efficient Xbar-based Computing Abhishek Moitra Abhiroop Bhattacharjee Youngeun Kim Priyadarshini Panda ViT 71 2 0 22 Aug 2024
From Efficient Multimodal Models to World Models: A Survey Xinji Mai Zeng Tao Junxiong Lin Haoran Wang Yang Chang Yanlan Kang Yan Wang Wenqiang Zhang 93 6 0 27 Jun 2024
Adaptively Bypassing Vision Transformer Blocks for Efficient Visual Tracking Xiangyang Yang Dan Zeng Xucheng Wang You Wu Hengzhou Ye Qijun Zhao Shuiwang Li 108 4 0 12 Jun 2024
Efficient Multimodal Large Language Models: A Survey Yizhang Jin Jian Li Yexin Liu Tianjun Gu Kai Wu ... Xin Tan Zhenye Gan Yabiao Wang Chengjie Wang Lizhuang Ma LRM 119 58 0 17 May 2024
Exploring Learngene via Stage-wise Weight Sharing for Initializing Variable-sized Models Shiyu Xia Wenxuan Zhu Xu Yang Xin Geng 61 2 0 25 Apr 2024
Data-independent Module-aware Pruning for Hierarchical Vision Transformers Yang He Qiufeng Wang ViT 83 5 0 21 Apr 2024
Weight Copy and Low-Rank Adaptation for Few-Shot Distillation of Vision Transformers Diana-Nicoleta Grigore Mariana-Iuliana Georgescu J. A. Justo T. Johansen Andreea-Iuliana Ionescu Radu Tudor Ionescu 81 0 0 14 Apr 2024
Dense Vision Transformer Compression with Few Samples Hanxiao Zhang Yifan Zhou Guo-Hua Wang Jianxin Wu ViT VLM 84 4 0 27 Mar 2024
GRITv2: Efficient and Light-weight Social Relation Recognition Sagar Reddy Neeraj Kasera Avinash Thakur ViT 60 0 0 11 Mar 2024
On the Convergence of Differentially-Private Fine-tuning: To Linearly Probe or to Fully Fine-tune? Shuqi Ke Charlie Hou Giulia Fanti Sewoong Oh 83 4 0 29 Feb 2024
Understanding Neural Network Binarization with Forward and Backward Proximal Quantizers Yiwei Lu Yaoliang Yu Xinlin Li Vahid Partovi Nia MQ 82 3 0 27 Feb 2024
EffLoc: Lightweight Vision Transformer for Efficient 6-DOF Camera Relocalization Zhendong Xiao Changhao Chen Shan Yang Wu Wei 53 1 0 21 Feb 2024
Head-wise Shareable Attention for Large Language Models Zouying Cao Yifei Yang Hai Zhao 71 4 0 19 Feb 2024
LRP-QViT: Mixed-Precision Vision Transformer Quantization via Layer-wise Relevance Propagation Navin Ranjan Andreas E. Savakis MQ 97 6 0 20 Jan 2024
Group Multi-View Transformer for 3D Shape Analysis with Spatial Encoding Lixiang Xu Qingzhe Cui Richang Hong Wei Xu Enhong Chen Xin Yuan Chenglong Li Y. Tang 89 2 0 27 Dec 2023
Transformer as Linear Expansion of Learngene Shiyu Xia Miaosen Zhang Xu Yang Ruiming Chen Haokun Chen Xin Geng 75 7 0 09 Dec 2023
PhytNet -- Tailored Convolutional Neural Networks for Custom Botanical Data Jamie R. Sykes Katherine Denby Daniel W. Franks 61 2 0 20 Nov 2023
I&S-ViT: An Inclusive & Stable Method for Pushing the Limit of Post-Training ViTs Quantization Mingliang Xu Jiawei Hu Mingbao Lin Mengzhao Chen Rongrong Ji MQ 88 11 0 16 Nov 2023
Lightweight Full-Convolutional Siamese Tracker Yunfeng Li Bo Wang Xueyi Wu Zhuoyan Liu Ye Li 80 10 0 09 Oct 2023
CAIT: Triple-Win Compression towards High Accuracy, Fast Inference, and Favorable Transferability For ViTs Ao Wang Hui Chen Zijia Lin Sicheng Zhao Jiawei Han Guiguang Ding ViT 58 6 0 27 Sep 2023
A survey on efficient vision transformers: algorithms, techniques, and performance benchmarking Lorenzo Papa Paolo Russo Irene Amerini Luping Zhou 100 46 0 05 Sep 2023
Revisiting Vision Transformer from the View of Path Ensemble Shuning Chang Pichao Wang Haowen Luo Fan Wang Mike Zheng Shou ViT 64 3 0 12 Aug 2023
A Good Student is Cooperative and Reliable: CNN-Transformer Collaborative Learning for Semantic Segmentation Jinjing Zhu Yuan Luo Xueye Zheng Hao Wang Lin Wang 65 35 0 24 Jul 2023
Revisiting Token Pruning for Object Detection and Instance Segmentation Yifei Liu Mathias Gehrig Nico Messikommer Marco Cannici Davide Scaramuzza ViT VLM 112 27 0 12 Jun 2023
MixFormerV2: Efficient Fully Transformer Tracking Yutao Cui Tian-Shu Song Gangshan Wu Liming Wang 90 59 0 25 May 2023
Fast-StrucTexT: An Efficient Hourglass Transformer with Modality-guided Dynamic Token Merge for Document Understanding Mingliang Zhai Yulin Li Xiameng Qin Chen Yi Qunyi Xie Chengquan Zhang Kun Yao Yuwei Wu Yunde Jia 42 8 0 19 May 2023
Boost Vision Transformer with GPU-Friendly Sparsity and Quantization Chong Yu Tao Chen Zhongxue Gan Jiayuan Fan MQ ViT 78 25 0 18 May 2023
EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attention Xinyu Liu Houwen Peng Ningxin Zheng Yuqing Yang Han Hu Yixuan Yuan ViT 93 316 0 11 May 2023
Patch-wise Mixed-Precision Quantization of Vision Transformer Junrui Xiao Zhikai Li Lianwei Yang Qingyi Gu MQ 89 12 0 11 May 2023
RIFormer: Keep Your Vision Backbone Effective While Removing Token Mixer Jiahao Wang Songyang Zhang Yong Liu Taiqiang Wu Yujiu Yang Xihui Liu Kai-xiang Chen Ping Luo Dahua Lin 79 22 0 12 Apr 2023
Sim-T: Simplify the Transformer Network by Multiplexing Technique for Speech Recognition Guangyong Wei Zhikui Duan Shiren Li Guangguang Yang Xinmei Yu Junhua Li 68 5 0 11 Apr 2023
Towards Efficient Task-Driven Model Reprogramming with Foundation Models Shoukai Xu Jiangchao Yao Ran Luo Shuhai Zhang Zihao Lian Mingkui Tan Bo Han Yaowei Wang 95 6 0 05 Apr 2023
Scaling Pre-trained Language Models to Deeper via Parameter-efficient Architecture Peiyu Liu Ze-Feng Gao Yushuo Chen Wayne Xin Zhao Ji-Rong Wen MoE 68 0 0 27 Mar 2023
Frame Flexible Network Yitian Zhang Yue Bai Chang Liu Huan Wang Sheng Li Yun Fu 47 4 0 26 Mar 2023
X-Pruner: eXplainable Pruning for Vision Transformers Lu Yu Wei Xiang ViT 60 50 0 08 Mar 2023
Rotation Invariant Quantization for Model Compression Dor-Joseph Kampeas Yury Nahshan Hanoch Kremer Gil Lederman Shira Zaloshinski Zheng Li E. Haleva MQ 119 1 0 03 Mar 2023
A Comprehensive Review and a Taxonomy of Edge Machine Learning: Requirements, Paradigms, and Techniques Wenbin Li Hakim Hacid Ebtesam Almazrouei Merouane Debbah 91 13 0 16 Feb 2023
Knowledge Distillation in Vision Transformers: A Critical Review Gousia Habib Tausifa Jan Saleem Brejesh Lall 98 16 0 04 Feb 2023
RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformers Zhikai Li Junrui Xiao Lianwei Yang Qingyi Gu MQ 84 90 0 16 Dec 2022