Sparse-Tuning: Adapting Vision Transformers with Efficient Fine-tuning and Inference

23 May 2024

Ting Liu

Xuyang Liu

Liangtao Shi

Zunnan Xu

Siteng Huang

Yi Xin

Quanjun Yin

ArXiv PDF HTML

Papers citing "Sparse-Tuning: Adapting Vision Transformers with Efficient Fine-tuning and Inference"

32 / 32 papers shown

Title
KARST: Multi-Kernel Kronecker Adaptation with Re-Scaling Transmission for Visual Classification Yue Zhu Haiwen Diao Shang Gao Long Chen Huchuan Lu 124 0 0 10 Feb 2025
MaPPER: Multimodal Prior-guided Parameter Efficient Tuning for Referring Expression Comprehension Ting Liu Zunnan Xu Yue Hu Liangtao Shi Zhiqiang Wang Quanjun Yin 104 3 0 03 Jan 2025
Parameter-Efficient and Memory-Efficient Tuning for Vision Transformer: A Disentangled Approach Taolin Zhang Jiawang Bai Zhihe Lu Dongze Lian Genping Wang Xinchao Wang Shu-Tao Xia 80 5 0 09 Jul 2024
DARA: Domain- and Relation-aware Adapters Make Parameter-efficient Tuning for Visual Grounding Ting Liu Xuyang Liu Siteng Huang Honggang Chen Quanjun Yin Long Qin Donglin Wang Yue Hu 60 6 0 10 May 2024
Time-, Memory- and Parameter-Efficient Visual Adaptation Otniel-Bogdan Mercea Alexey Gritsenko Cordelia Schmid Anurag Arnab VLM 53 14 0 05 Feb 2024
Parameter-Efficient Fine-Tuning for Pre-Trained Vision Models: A Survey Yi Xin Jianjiang Yang Haodi Zhou Junlong Du Junlong Du Yue Fan Qing Li Qing Li Yuntao Du VLM 101 81 0 03 Feb 2024
SparseViT: Revisiting Activation Sparsity for Efficient High-Resolution Vision Transformer Xuanyao Chen Zhijian Liu Haotian Tang Li Yi Hang Zhao Song Han ViT 176 48 0 30 Mar 2023
FacT: Factor-Tuning for Lightweight Adaptation on Vision Transformer Shibo Jie Zhi-Hong Deng 49 131 0 06 Dec 2022
Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuning Dongze Lian Daquan Zhou Jiashi Feng Xinchao Wang 74 258 0 17 Oct 2022
Convolutional Bypasses Are Better Vision Transformer Adapters Shibo Jie Zhi-Hong Deng VPVLM 59 133 0 14 Jul 2022
ST-Adapter: Parameter-Efficient Image-to-Video Transfer Learning Junting Pan Ziyi Lin Xiatian Zhu Jing Shao Hongsheng Li 82 203 0 27 Jun 2022
Neural Prompt Search Yuanhan Zhang Kaiyang Zhou Ziwei Liu VPVLM VLM 93 149 0 09 Jun 2022
AdapterBias: Parameter-efficient Token-dependent Representation Shift for Adapters in NLP Tasks Chin-Lun Fu Zih-Ching Chen Yun-Ru Lee Hung-yi Lee 60 47 0 30 Apr 2022
Visual Prompt Tuning Menglin Jia Luming Tang Bor-Chun Chen Claire Cardie Serge Belongie Bharath Hariharan Ser-Nam Lim VLM VPVLM 148 1,624 0 23 Mar 2022
Swin Transformer V2: Scaling Up Capacity and Resolution Ze Liu Han Hu Yutong Lin Zhuliang Yao Zhenda Xie ... Yue Cao Zheng Zhang Li Dong Furu Wei B. Guo ViT 207 1,809 0 18 Nov 2021
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 451 7,739 0 11 Nov 2021
BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models Elad Ben-Zaken Shauli Ravfogel Yoav Goldberg 162 1,218 0 18 Jun 2021
DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification Yongming Rao Wenliang Zhao Benlin Liu Jiwen Lu Jie Zhou Cho-Jui Hsieh ViT 78 697 0 03 Jun 2021
Not All Images are Worth 16x16 Words: Dynamic Transformers for Efficient Image Recognition Yulin Wang Rui Huang S. Song Zeyi Huang Gao Huang ViT 71 193 0 31 May 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 668 6,066 0 29 Apr 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 543 4,036 0 18 Apr 2021
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows Ze Liu Yutong Lin Yue Cao Han Hu Yixuan Wei Zheng Zhang Stephen Lin B. Guo ViT 441 21,418 0 25 Mar 2021
Dynamic Neural Networks: A Survey Yizeng Han Gao Huang Shiji Song Le Yang Honghui Wang Yulin Wang 3DH AI4TS AI4CE 109 649 0 09 Feb 2021
A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmark Xiaohua Zhai J. Puigcerver Alexander Kolesnikov P. Ruyssen C. Riquelme ... Michael Tschannen Marcin Michalski Olivier Bousquet Sylvain Gelly N. Houlsby SSL 79 439 0 01 Oct 2019
Do Better ImageNet Models Transfer Better? Simon Kornblith Jonathon Shlens Quoc V. Le OOD MLT 155 1,327 0 23 May 2018
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset João Carreira Andrew Zisserman 229 8,015 0 22 May 2017
Densely Connected Convolutional Networks Gao Huang Zhuang Liu Laurens van der Maaten Kilian Q. Weinberger PINN 3DV 766 36,794 0 25 Aug 2016
SGDR: Stochastic Gradient Descent with Warm Restarts I. Loshchilov Frank Hutter ODL 327 8,116 0 13 Aug 2016
Layer Normalization Jimmy Lei Ba J. Kiros Geoffrey E. Hinton 410 10,482 0 21 Jul 2016
Gaussian Error Linear Units (GELUs) Dan Hendrycks Kevin Gimpel 169 5,000 0 27 Jun 2016
Microsoft COCO: Common Objects in Context Nayeon Lee Michael Maire Serge J. Belongie Lubomir Bourdev Ross B. Girshick James Hays Pietro Perona Deva Ramanan C. L. Zitnick Piotr Dollár ObjD 413 43,638 0 01 May 2014
Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks Ian Goodfellow Yaroslav Bulatov Julian Ibarz Sacha Arnoud Vinay D. Shet 95 719 0 20 Dec 2013