CvT: Introducing Convolutions to Vision Transformers

29 March 2021

Lu Yuan

Lei Zhang

ViT

ArXiv PDF HTML

Papers citing "CvT: Introducing Convolutions to Vision Transformers"

50 / 819 papers shown

Title
Rethinking Local Perception in Lightweight Vision Transformer Qi Fan Huaibo Huang Jiyang Guan Ran He ViT 31 30 0 31 Mar 2023
Dual Cross-Attention for Medical Image Segmentation Gorkem Can Ates P. Mohan Emrah Çelik 17 76 0 30 Mar 2023
TFS-ViT: Token-Level Feature Stylization for Domain Generalization Mehrdad Noori Milad Cheraghalikhani Ali Bahri G. A. V. Hakim David Osowiechi Ismail Ben Ayed Christian Desrosiers 25 10 0 28 Mar 2023
MoViT: Memorizing Vision Transformers for Medical Image Analysis Yiqing Shen Pengfei Guo Jinpu Wu Qi Huang Nhat Le Jinyuan Zhou Shanshan Jiang Mathias Unberath ViT MedIm 34 10 0 27 Mar 2023
Supervised Masked Knowledge Distillation for Few-Shot Transformers Hanxi Lin G. Han Jiawei Ma Shiyuan Huang Xudong Lin Shih-Fu Chang 24 35 0 25 Mar 2023
FastViT: A Fast Hybrid Vision Transformer using Structural Reparameterization Pavan Kumar Anasosalu Vasu J. Gabriel Jeff J. Zhu Oncel Tuzel Anurag Ranjan ViT 37 155 0 24 Mar 2023
Exploiting Unlabelled Photos for Stronger Fine-Grained SBIR Aneeshan Sain A. Bhunia Subhadeep Koley Pinaki Nath Chowdhury Soumitri Chattopadhyay Tao Xiang Yi-Zhe Song 30 18 0 24 Mar 2023
FER-former: Multi-modal Transformer for Facial Expression Recognition Yande Li Mingjie Wang Minglun Gong Y. Lu Li Liu 30 8 0 23 Mar 2023
Machine Learning for Brain Disorders: Transformers and Visual Transformers Robin Courant Maika Edberg Nicolas Dufour Vicky Kalogeiton MedIm ViT 40 1 0 21 Mar 2023
Sparse-IFT: Sparse Iso-FLOP Transformations for Maximizing Training Efficiency Vithursan Thangarasa Shreyas Saxena Abhay Gupta Sean Lie 41 3 0 21 Mar 2023
Convolutions, Transformers, and their Ensembles for the Segmentation of Organs at Risk in Radiation Treatment of Cervical Cancer Vangelis Kostoulas Peter A. N. Bosman Tanja Alderliesten UQCV 28 1 0 20 Mar 2023
Robustifying Token Attention for Vision Transformers Yong Guo David Stutz Bernt Schiele ViT 23 24 0 20 Mar 2023
Tracker Meets Night: A Transformer Enhancer for UAV Tracking Junjie Ye Changhong Fu Ziang Cao Shan An Guang-Zheng Zheng Bowen Li 37 52 0 20 Mar 2023
Deephys: Deep Electrophysiology, Debugging Neural Networks under Distribution Shifts Anirban Sarkar Matthew Groth I. Mason Tomotake Sasaki Xavier Boix 21 1 0 17 Mar 2023
Dual-path Adaptation from Image to Video Transformers Jungin Park Jiyoung Lee Kwanghoon Sohn ViT 21 37 0 17 Mar 2023
ElasticViT: Conflict-aware Supernet Training for Deploying Fast Vision Transformer on Diverse Mobile Devices Chen Tang Li Zhang Huiqiang Jiang Jiahang Xu Ting Cao Quanlu Zhang Yuqing Yang Zhi Wang Mao Yang 28 11 0 17 Mar 2023
Making Vision Transformers Efficient from A Token Sparsification View Shuning Chang Pichao Wang Ming Lin Fan Wang David Junhao Zhang Rong Jin Mike Zheng Shou ViT 50 24 0 15 Mar 2023
CrossFormer++: A Versatile Vision Transformer Hinging on Cross-scale Attention Wenxiao Wang Wei Chen Qibo Qiu Long Chen Boxi Wu Binbin Lin Xiaofei He Wei Liu 35 39 0 13 Mar 2023
Neighborhood Contrastive Transformer for Change Captioning Yunbin Tu Liang Li Li Su Kelvin Lu Qin Huang ViT 24 14 0 06 Mar 2023
Retinal Image Restoration using Transformer and Cycle-Consistent Generative Adversarial Network Alnur Alimanov Md Baharul Islam ViT MedIm 27 4 0 03 Mar 2023
Self-attention in Vision Transformers Performs Perceptual Grouping, Not Attention Paria Mehrani John K. Tsotsos 25 24 0 02 Mar 2023
OmniForce: On Human-Centered, Large Model Empowered and Cloud-Edge Collaborative AutoML System Chao Xue Wen Liu Shunxing Xie Zhenfang Wang Jiaxing Li ... Shi-Yong Chen Yibing Zhan Jing Zhang Chaoyue Wang Dacheng Tao 52 2 0 01 Mar 2023
Extracting Motion and Appearance via Inter-Frame Attention for Efficient Video Frame Interpolation Guozhen Zhang Yuhan Zhu Hongya Wang Youxin Chen Gangshan Wu Limin Wang 74 85 0 01 Mar 2023
Enhancing Classification with Hierarchical Scalable Query on Fusion Transformer S. K. Sahoo Sathish Chalasani Abhishek Joshi K. N. Iyer 33 2 0 28 Feb 2023
A Convolutional Vision Transformer for Semantic Segmentation of Side-Scan Sonar Data Hayat Rajani N. Gracias Rafael García ViT 27 12 0 24 Feb 2023
Transformers in Single Object Tracking: An Experimental Survey Janani Kugarajeevan T. Kokul A. Ramanan Subha Fernando 38 35 0 23 Feb 2023
Human MotionFormer: Transferring Human Motions with Vision Transformers Hongyu Liu Xintong Han Chengbin Jin Lihui Qian Huawei Wei ... Faqiang Wang Haoye Dong Yibing Song Jia Xu Qifeng Chen 16 11 0 22 Feb 2023
Device Tuning for Multi-Task Large Model Penghao Jiang Xuanchen Hou Y. Zhou 26 0 0 21 Feb 2023
LIT-Former: Linking In-plane and Through-plane Transformers for Simultaneous CT Image Denoising and Deblurring Zhihao Chen Chuang Niu Qi Gao Ge Wang Hongming Shan MedIm ViT 3DV 46 20 0 21 Feb 2023
Soft Error Reliability Analysis of Vision Transformers Xing-xiong Xue Cheng Liu Ying Wang Bing Yang Yaoyu Zhang Lefei Zhang Huawei Li Xiaowei Li 39 14 0 21 Feb 2023
MedViT: A Robust Vision Transformer for Generalized Medical Image Classification Omid Nejati Manzari Hamid Ahmadabadi Hossein Kashiani S. B. Shokouhi Ahmad Ayatollahi ViT MedIm 34 179 0 19 Feb 2023
Efficiency 360: Efficient Vision Transformers Badri N. Patro Vijay Srinivas Agneeswaran 33 6 0 16 Feb 2023
TFormer: A Transmission-Friendly ViT Model for IoT Devices Zhichao Lu Chuntao Ding Felix Juefei Xu Vishnu Boddeti Shangguang Wang Yun Yang 28 13 0 15 Feb 2023
Robust Representation Learning with Self-Distillation for Domain Generalization Ankur Singh Senthilnath Jayavelu ViT OOD 18 2 0 14 Feb 2023
Towards Local Visual Modeling for Image Captioning Yiwei Ma Jiayi Ji Xiaoshuai Sun Yiyi Zhou Rongrong Ji ViT 21 71 0 13 Feb 2023
Efficient Attention via Control Variates Lin Zheng Jianbo Yuan Chong-Jun Wang Lingpeng Kong 34 18 0 09 Feb 2023
DilateFormer: Multi-Scale Dilated Transformer for Visual Recognition Jiayu Jiao Yuyao Tang Kun-Li Channing Lin Yipeng Gao Jinhua Ma Yaowei Wang Wei-Shi Zheng MedIm ViT 29 137 0 03 Feb 2023
Deep Dependency Networks for Multi-Label Classification Shivvrat Arya Yu Xiang Vibhav Gogate 16 0 0 01 Feb 2023
POSTER++: A simpler and stronger facial expression recognition network Jia-ju Mao Rui Xu Xuesong Yin Yuan Chang Binling Nie Aibin Huang CVBM 40 37 0 28 Jan 2023
Robust Transformer with Locality Inductive Bias and Feature Normalization Omid Nejati Manzari Hossein Kashiani Hojat Asgarian Dehkordi S. B. Shokouhi ViT 24 14 0 27 Jan 2023
Compact Transformer Tracker with Correlative Masked Modeling Zikai Song Run Luo Junqing Yu Yi-Ping Phoebe Chen Wei Yang ViT 30 57 0 26 Jan 2023
Exploiting Optical Flow Guidance for Transformer-Based Video Inpainting Kaiwen Zhang Jialun Peng Jingjing Fu Dong Liu ViT 29 8 0 24 Jan 2023
Gated-ViGAT: Efficient Bottom-Up Event Recognition and Explanation Using a New Frame Selection Policy and Gating Mechanism Nikolaos Gkalelis Dimitrios Daskalakis Vasileios Mezaris 24 4 0 18 Jan 2023
FGAHOI: Fine-Grained Anchors for Human-Object Interaction Detection Shuailei Ma Yuefeng Wang Shanze Wang Ying-yu Wei 53 33 0 08 Jan 2023
Filtering, Distillation, and Hard Negatives for Vision-Language Pre-Training Filip Radenovic Abhimanyu Dubey Abhishek Kadian Todor Mihaylov Simon Vandenhende Yash J. Patel Y. Wen Vignesh Ramanathan D. Mahajan VLM 40 82 0 05 Jan 2023
Skip-Attention: Improving Vision Transformers by Paying Less Attention Shashanka Venkataramanan Amir Ghodrati Yuki M. Asano Fatih Porikli A. Habibian ViT 23 25 0 05 Jan 2023
Rethinking Mobile Block for Efficient Attention-based Models Jiangning Zhang Xiangtai Li Jian Li Liang Liu Zhucun Xue Boshen Zhang Zhe Jiang Tianxin Huang Yabiao Wang Chengjie Wang MQ 49 91 0 03 Jan 2023
Local Learning on Transformers via Feature Reconstruction P. Pathak Jingwei Zhang Dimitris Samaras ViT 24 5 0 29 Dec 2022
Exploring Vision Transformers as Diffusion Learners He Cao Jianan Wang Tianhe Ren Xianbiao Qi Yihao Chen Yuan Yao Lefei Zhang 44 10 0 28 Dec 2022
A Close Look at Spatial Modeling: From Attention to Convolution Xu Ma Huan Wang Can Qin Kunpeng Li Xing Zhao Jie Fu Yun Fu ViT 3DPC 25 11 0 23 Dec 2022