CvT: Introducing Convolutions to Vision Transformers

29 March 2021

Lu Yuan

Lei Zhang

ViT

ArXiv PDF HTML

Papers citing "CvT: Introducing Convolutions to Vision Transformers"

50 / 818 papers shown

Title
ClST: A Convolutional Transformer Framework for Automatic Modulation Recognition by Knowledge Distillation Dongbin Hou Lixin Li Wensheng Lin Junli Liang Zhu Han 18 3 0 29 Dec 2023
Rethinking of Feature Interaction for Multi-task Learning on Dense Prediction Jingdong Zhang Jiayuan Fan Peng Ye Bo-Wen Zhang Hancheng Ye Baopu Li Yancheng Cai Tao Chen 22 2 0 21 Dec 2023
Cached Transformers: Improving Transformers with Differentiable Memory Cache Zhaoyang Zhang Wenqi Shao Yixiao Ge Xiaogang Wang Liang Feng Ping Luo 16 2 0 20 Dec 2023
ConDaFormer: Disassembled Transformer with Local Structure Enhancement for 3D Point Cloud Understanding Lunhao Duan Shanshan Zhao Nan Xue Biwei Huang Gui-Song Xia Dacheng Tao ViT 35 18 0 18 Dec 2023
Agent Attention: On the Integration of Softmax and Linear Attention Dongchen Han Tianzhu Ye Yizeng Han Zhuofan Xia Siyuan Pan Pengfei Wan Shiji Song Gao Huang 34 74 0 14 Dec 2023
Transformer-based Selective Super-Resolution for Efficient Image Refinement Tianyi Zhang Kishore Kasichainula Yaoxin Zhuo Baoxin Li Jae-sun Seo Yu Cao 26 7 0 10 Dec 2023
Graph Convolutions Enrich the Self-Attention in Transformers! Jeongwhan Choi Hyowon Wi Jayoung Kim Yehjin Shin Kookjin Lee Nathaniel Trask Noseong Park 32 4 0 07 Dec 2023
Class-Discriminative Attention Maps for Vision Transformers L. Brocki Jakub Binda N. C. Chung MedIm 32 3 0 04 Dec 2023
MobileUtr: Revisiting the relationship between light-weight CNN and Transformer for efficient medical image segmentation Fenghe Tang Bingkun Nian Jianrui Ding Quan Quan Jie-jin Yang Wei Liu S.Kevin Zhou ViT MedIm 23 3 0 04 Dec 2023
SCHEME: Scalable Channel Mixer for Vision Transformers Deepak Sridhar Yunsheng Li Nuno Vasconcelos 47 0 0 01 Dec 2023
TransNeXt: Robust Foveal Visual Perception for Vision Transformers Dai Shi ViT 23 76 0 28 Nov 2023
Beyond Visual Cues: Synchronously Exploring Target-Centric Semantics for Vision-Language Tracking Jiawei Ge Xiangmei Chen Jiuxin Cao Xueling Zhu Bo Liu VLM 44 2 0 28 Nov 2023
Advancing Vision Transformers with Group-Mix Attention Chongjian Ge Xiaohan Ding Zhan Tong Li Yuan Jiangliu Wang Yibing Song Ping Luo 112 16 0 26 Nov 2023
Pursing the Sparse Limitation of Spiking Deep Learning Structures Hao-Ran Cheng Jiahang Cao Erjia Xiao Mengshu Sun Le Yang Jize Zhang Xue Lin B. Kailkhura Kaidi Xu Renjing Xu 16 1 0 18 Nov 2023
Vision Big Bird: Random Sparsification for Full Attention Zhemin Zhang Xun Gong ViT 13 1 0 10 Nov 2023
Mini but Mighty: Finetuning ViTs with Mini Adapters Imad Eddine Marouf Enzo Tartaglione Stéphane Lathuilière 36 5 0 07 Nov 2023
GTP-ViT: Efficient Vision Transformers via Graph-based Token Propagation Xuwei Xu Sen Wang Yudong Chen Yanping Zheng Zhewei Wei Jiajun Liu ViT 27 8 0 06 Nov 2023
Dense Video Captioning: A Survey of Techniques, Datasets and Evaluation Protocols Iqra Qasim Alexander Horsch Dilip K. Prasad 22 6 0 05 Nov 2023
Scattering Vision Transformer: Spectral Mixing Matters Badri N. Patro Vijay Srinivas Agneeswaran 37 14 0 02 Nov 2023
Distilling Knowledge from CNN-Transformer Models for Enhanced Human Action Recognition Hamid Ahmadabadi Omid Nejati Manzari Ahmad Ayatollahi 21 7 0 02 Nov 2023
Limited Data, Unlimited Potential: A Study on ViTs Augmented by Masked Autoencoders Srijan Das Tanmay Jain Dominick Reilly P. Balaji Soumyajit Karmakar Shyam Marjit Xiang Li Abhijit Das Michael S. Ryoo 39 16 0 31 Oct 2023
MIST: Medical Image Segmentation Transformer with Convolutional Attention Mixing (CAM) Decoder Md Motiur Rahman Shiva Shokouhmand Smriti Bhatt M. Faezipour MedIm 42 15 0 30 Oct 2023
ViR: Towards Efficient Vision Retention Backbones Ali Hatamizadeh Michael Ranzinger Shiyi Lan Jose M. Alvarez Sanja Fidler Jan Kautz GNN 22 1 0 30 Oct 2023
AViTMP: A Tracking-Specific Transformer for Single-Branch Visual Tracking Chuanming Tang Kai Wang Joost van de Weijer Jianlin Zhang Yongmei Huang 23 0 0 30 Oct 2023
TransXNet: Learning Both Global and Local Dynamics with a Dual Dynamic Token Mixer for Visual Recognition Meng Lou Hong-Yu Zhou Sibei Yang Yizhou Yu Chuan Wu Yizhou Yu ViT 44 36 0 30 Oct 2023
Exploring Shape Embedding for Cloth-Changing Person Re-Identification via 2D-3D Correspondences Yubin Wang Huimin Yu Yuming Yan Shuyi Song Biyang Liu Yichong Lu 3DPC 24 7 0 27 Oct 2023
Generalizing to Unseen Domains in Diabetic Retinopathy Classification Chamuditha Jayanga Galappaththige Gayal Kuruppu Muhammad Haris Khan OOD 24 7 0 26 Oct 2023
Bridging The Gaps Between Token Pruning and Full Pre-training via Masked Fine-tuning Fengyuan Shi Limin Wang ViT 38 0 0 26 Oct 2023
Toward Flare-Free Images: A Survey Yousef Kotp Marwan Torki 46 3 0 22 Oct 2023
Multimodal Transformer Using Cross-Channel attention for Object Detection in Remote Sensing Images Bissmella Bahaduri Zuheng Ming Fangchen Feng Anissa Mokraou 32 1 0 21 Oct 2023
LeTFuser: Light-weight End-to-end Transformer-Based Sensor Fusion for Autonomous Driving with Multi-Task Learning Pedram Agand Mohammad Mahdavian Manolis Savva Mo Chen ViT 29 4 0 19 Oct 2023
Minimalist and High-Performance Semantic Segmentation with Plain Vision Transformers Yuanduo Hong Jue Wang Weichao Sun Huihui Pan VLM ViT 40 7 0 19 Oct 2023
Camera-LiDAR Fusion with Latent Contact for Place Recognition in Challenging Cross-Scenes Yan Pan Jiapeng Xie Jiajie Wu Bo Zhou 36 0 0 16 Oct 2023
Accelerating Vision Transformers Based on Heterogeneous Attention Patterns Deli Yu Teng Xi Jianwei Li Baopu Li Gang Zhang Haocheng Feng Junyu Han Jingtuo Liu Errui Ding Jingdong Wang ViT 31 0 0 11 Oct 2023
Distance Weighted Trans Network for Image Completion Pourya Shamsolmoali Masoumeh Zareapoor Huiyu Zhou Xuelong Li Yue Lu ViT 33 0 0 11 Oct 2023
Distilling Efficient Vision Transformers from CNNs for Semantic Segmentation Xueye Zheng Yunhao Luo Pengyuan Zhou Lin Wang 35 13 0 11 Oct 2023
EViT: An Eagle Vision Transformer with Bi-Fovea Self-Attention Yulong Shi Mingwei Sun Yongshuai Wang Hui Sun Zengqiang Chen 34 4 0 10 Oct 2023
No Token Left Behind: Efficient Vision Transformer via Dynamic Token Idling Xuwei Xu Changlin Li Yudong Chen Xiaojun Chang Jiajun Liu Sen Wang ViT 21 5 0 09 Oct 2023
Plug n' Play: Channel Shuffle Module for Enhancing Tiny Vision Transformers Xuwei Xu Sen Wang Yudong Chen Jiajun Liu ViT 21 1 0 09 Oct 2023
Enhancing Representations through Heterogeneous Self-Supervised Learning Zhongyu Li Bo-Wen Yin Yongxiang Liu Li Liu Ming-Ming Cheng SSL 28 2 0 08 Oct 2023
Low-Resolution Self-Attention for Semantic Segmentation Yu-Huan Wu Shi-Chen Zhang Yun-Hai Liu Le Zhang Xin Zhan Daquan Zhou Jiashi Feng Ming-Ming Cheng Liangli Zhen ViT 45 3 0 08 Oct 2023
TiC: Exploring Vision Transformer in Convolution Song Zhang Qingzhong Wang Jiang Bian Haoyi Xiong ViT 31 1 0 06 Oct 2023
ClusVPR: Efficient Visual Place Recognition with Clustering-based Weighted Transformer Yifan Xu Pourya Shamsolmoali Jie Yang ViT 23 1 0 06 Oct 2023
TransRadar: Adaptive-Directional Transformer for Real-Time Multi-View Radar Semantic Segmentation Yahia Dalbah Jean Lahoud Hisham Cholakkal 19 7 0 03 Oct 2023
Towards Training Without Depth Limits: Batch Normalization Without Gradient Explosion Alexandru Meterez Amir Joudaki Francesco Orabona Alexander Immer Gunnar Rätsch Hadi Daneshmand 32 8 0 03 Oct 2023
Understanding Masked Autoencoders From a Local Contrastive Perspective Xiaoyu Yue Lei Bai Meng Wei Jiangmiao Pang Xihui Liu Luping Zhou Wanli Ouyang SSL 67 4 0 03 Oct 2023
PPT: Token Pruning and Pooling for Efficient Vision Transformers Xinjian Wu Fanhu Zeng Xiudong Wang Xinghao Chen ViT 32 22 0 03 Oct 2023
SeisT: A foundational deep learning model for earthquake monitoring tasks Sen Li Xu Yang Anye Cao Changbin Wang Yaoqi Liu Yapeng Liu Qiang Niu 33 3 0 02 Oct 2023
Win-Win: Training High-Resolution Vision Transformers from Two Windows Vincent Leroy Jérôme Revaud Thomas Lucas Philippe Weinzaepfel ViT 42 2 0 01 Oct 2023
RBFormer: Improve Adversarial Robustness of Transformer by Robust Bias Hao Cheng Jinhao Duan Hui Li Lyutianyang Zhang Jiahang Cao Ping Wang Jize Zhang Kaidi Xu Renjing Xu AAML 32 3 0 23 Sep 2023