CvT: Introducing Convolutions to Vision Transformers

29 March 2021

Lu Yuan

Lei Zhang

ViT

ArXiv PDF HTML

Papers citing "CvT: Introducing Convolutions to Vision Transformers"

50 / 819 papers shown

Title
Self-Distilled Masked Auto-Encoders are Efficient Video Anomaly Detectors Nicolae-Cătălin Ristea Florinel-Alin Croitoru Radu Tudor Ionescu Marius Popescu Fahad Shahbaz Khan M. Shah ViT 26 20 0 21 Jun 2023
Understanding Optimization of Deep Learning via Jacobian Matrix and Lipschitz Constant Xianbiao Qi Jianan Wang Lei Zhang 21 0 0 15 Jun 2023
Exploring the Application of Large-scale Pre-trained Models on Adverse Weather Removal Zhentao Tan Yue-bo Wu Qiankun Liu Qi Chu Le Lu Jieping Ye Nenghai Yu 48 11 0 15 Jun 2023
Is Anisotropy Inherent to Transformers? Nathan Godey Eric Villemonte de la Clergerie Benoît Sagot 19 3 0 13 Jun 2023
Sea Ice Segmentation From SAR Data by Convolutional Transformer Networks Nicolae-Cătălin Ristea Andrei Anghel Mihai Datcu ViT 8 4 0 13 Jun 2023
Speaker Embeddings as Individuality Proxy for Voice Stress Detection Zihan Wu Neil Scheidwasser Karl El Hajal Milos Cernak 32 3 0 09 Jun 2023
Multi-Architecture Multi-Expert Diffusion Models Yunsung Lee Jin-Young Kim Hyojun Go Myeongho Jeong Shinhyeok Oh Seungtaek Choi DiffM 39 30 0 08 Jun 2023
Multi-Scale And Token Mergence: Make Your ViT More Efficient Zhe Bian Zhe Wang Wenqiang Han Kangping Wang 25 5 0 08 Jun 2023
InvPT++: Inverted Pyramid Multi-Task Transformer for Visual Scene Understanding Hanrong Ye Dan Xu ViT 40 10 0 08 Jun 2023
TEC-Net: Vision Transformer Embrace Convolutional Neural Networks for Medical Image Segmentation Rui Sun Tao Lei Xiaogang Du Y. Wan Yong-quan Xia A. Nandi ViT MedIm 14 5 0 07 Jun 2023
CiT-Net: Convolutional Neural Networks Hand in Hand with Vision Transformers for Medical Image Segmentation Tao Lei Rui Sun Xuan Wang Yingbo Wang Xi He A. Nandi ViT MedIm 22 12 0 06 Jun 2023
Lightweight Structure-aware Transformer Network for VHR Remote Sensing Image Change Detection Tao Lei Yetong Xu Hailong Ning Z. Lv Chongdan Min Yaochu Jin A. Nandi ViT 11 6 0 03 Jun 2023
Recent Advances of Local Mechanisms in Computer Vision: A Survey and Outlook of Recent Work Qiangchang Wang Yilong Yin 48 0 0 02 Jun 2023
DeepFake-Adapter: Dual-Level Adapter for DeepFake Detection Rui Shao Tianxing Wu Liqiang Nie Ziwei Liu 34 11 0 01 Jun 2023
Lightweight Vision Transformer with Bidirectional Interaction Qihang Fan Huaibo Huang Xiaoqiang Zhou Ran He ViT 57 28 0 01 Jun 2023
AMatFormer: Efficient Feature Matching via Anchor Matching Transformer Bo Jiang S. Luo Tianlin Li Chuanfu Li Jin Tang 38 8 0 30 May 2023
Caterpillar: A Pure-MLP Architecture with Shifted-Pillars-Concatenation J. Sun Xiaoshuang Shi Zhiyuan Weng Kaidi Xu H. Shen Xiao-lan Zhu MLLM 33 2 0 28 May 2023
Making Vision Transformers Truly Shift-Equivariant Renan A. Rojas-Gomez Teck-Yian Lim Minh N. Do Raymond A. Yeh ViT 43 7 0 25 May 2023
Dual Path Transformer with Partition Attention Zhengkai Jiang Liang Liu Jiangning Zhang Yabiao Wang Mingang Chen Chengjie Wang ViT 38 2 0 24 May 2023
PanoContext-Former: Panoramic Total Scene Understanding with a Transformer Yuan Dong C. Fang Liefeng Bo Zilong Dong Ping Tan MDE ViT 23 11 0 21 May 2023
Efficient Mixed Transformer for Single Image Super-Resolution Ling Zheng Jinchen Zhu Jinpeng Shi Shizhuang Weng 43 19 0 19 May 2023
Two-Stream Regression Network for Dental Implant Position Prediction Xinquan Yang Xuguang Li Xuechen Li Wenting Chen Linlin Shen Xuran Li Yongqiang Deng 38 6 0 17 May 2023
CageViT: Convolutional Activation Guided Efficient Vision Transformer Hao Zheng Jinbao Wang Xiantong Zhen Hao Chen Jingkuan Song Feng Zheng ViT 32 0 0 17 May 2023
Mimetic Initialization of Self-Attention Layers Asher Trockman J. Zico Kolter 30 31 0 16 May 2023
CB-HVTNet: A channel-boosted hybrid vision transformer network for lymphocyte assessment in histopathological images Momina Liaqat Ali Zunaira Rauf Asifullah Khan A. Sohail Rafi Ullah Jeonghwan Gwak MedIm ViT 36 2 0 16 May 2023
Low-Light Image Enhancement via Structure Modeling and Guidance Xiaogang Xu Jiafei Wu Jiangbo Lu 3DV DiffM 27 67 0 10 May 2023
Dual flow fusion model for concrete surface crack segmentation Yuwei Duan 19 1 0 09 May 2023
Cross-Shaped Windows Transformer with Self-supervised Pretraining for Clinically Significant Prostate Cancer Detection in Bi-parametric MRI Yuheng Li Jacob F. Wynne Jing Wang Richard L. J. Qiu J. Roper ... A. Jani Tian Liu P. Patel H. Mao Xiaofeng Yang OOD ViT MedIm 35 10 0 30 Apr 2023
DIAMANT: Dual Image-Attention Map Encoders For Medical Image Segmentation Yousef Yeganeh Azade Farshad Peter Weinberger Seyed-Ahmad Ahmadi Ehsan Adeli Nassir Navab ViT MedIm 33 0 0 28 Apr 2023
Vision Conformer: Incorporating Convolutions into Vision Transformer Layers Brian Kenji Iwana Akihiro Kusuda ViT 45 2 0 27 Apr 2023
ScatterFormer: Locally-Invariant Scattering Transformer for Patient-Independent Multispectral Detection of Epileptiform Discharges Rui-Hua Zheng Jun Yu Li Yi Wang Tian Luo Yuguo Yu MedIm 40 4 0 26 Apr 2023
Hint-Aug: Drawing Hints from Foundation Vision Transformers Towards Boosted Few-Shot Parameter-Efficient Tuning Zhongzhi Yu Shang Wu Y. Fu Shunyao Zhang Yingyan Lin 33 6 0 25 Apr 2023
Universal Domain Adaptation via Compressive Attention Matching Didi Zhu Yincuan Li Junkun Yuan Zexi Li Kun Kuang Chao Wu 47 17 0 24 Apr 2023
TransFlow: Transformer as Flow Learner Yawen Lu Qifan Wang Siqi Ma Tong Geng Victor Y. Chen Huaijin Chen Dongfang Liu ViT 42 45 0 23 Apr 2023
Dilated-UNet: A Fast and Accurate Medical Image Segmentation Approach using a Dilated Transformer and U-Net Architecture Davoud Saadati Omid Nejati Manzari S. Mirzakuchaki ViT MedIm 36 15 0 22 Apr 2023
Two Birds, One Stone: A Unified Framework for Joint Learning of Image and Video Style Transfers Bohai Gu Heng Fan Libo Zhang 34 9 0 22 Apr 2023
Joint Token Pruning and Squeezing Towards More Aggressive Compression of Vision Transformers Siyuan Wei Tianzhu Ye Shen Zhang Yao Tang Jiajun Liang ViT 11 65 0 21 Apr 2023
LipsFormer: Introducing Lipschitz Continuity to Vision Transformers Xianbiao Qi Jianan Wang Yihao Chen Yukai Shi Lei Zhang 46 17 0 19 Apr 2023
Transformer-Based Visual Segmentation: A Survey Xiangtai Li Henghui Ding Haobo Yuan Wenwei Zhang Jiangmiao Pang Guangliang Cheng Kai-xiang Chen Ziwei Liu Chen Change Loy ViT MedIm 42 133 0 19 Apr 2023
DisCo-CLIP: A Distributed Contrastive Loss for Memory Efficient CLIP Training Yihao Chen Xianbiao Qi Jianan Wang Lei Zhang 23 16 0 17 Apr 2023
Deep-Learning-based Vasculature Extraction for Single-Scan Optical Coherence Tomography Angiography Jinpeng Liao Tianyu Zhang Yilong Zhang Chunhui Li Zhihong Huang 29 0 0 17 Apr 2023
SpectFormer: Frequency and Attention is what you need in a Vision Transformer Badri N. Patro Vinay P. Namboodiri Vijay Srinivas Agneeswaran ViT 40 48 0 13 Apr 2023
Dynamic Mobile-Former: Strengthening Dynamic Convolution with Attention and Residual Connection in Kernel Space Seokju Yun Youngmin Ro ViT 32 2 0 13 Apr 2023
DUFormer: Solving Power Line Detection Task in Aerial Images using Semantic Segmentation Deyu An Qian Zhang Jianshu Chao Tingli Li Feng Qiao Yong Deng Zhen-Peng Bian ViT 30 6 0 12 Apr 2023
PSLT: A Light-weight Vision Transformer with Ladder Self-Attention and Progressive Shift Gaojie Wu Weishi Zheng Yutong Lu Q. Tian ViT 48 15 0 07 Apr 2023
Visual Dependency Transformers: Dependency Tree Emerges from Reversed Attention Mingyu Ding Songlin Yang Lijie Fan Zhenfang Chen Z. Chen Ping Luo J. Tenenbaum Chuang Gan ViT 84 14 0 06 Apr 2023
ElegansNet: a brief scientific report and initial experiments Francesco Bardozzo Andrea Terlizzi Pietro Lio R. Tagliaferri 31 1 0 06 Apr 2023
MULLER: Multilayer Laplacian Resizer for Vision Zhengzhong Tu P. Milanfar Hossein Talebi 45 3 0 06 Apr 2023
Efficient CNNs via Passive Filter Pruning Arshdeep Singh Mark D. Plumbley 24 1 0 05 Apr 2023
Astroformer: More Data Might not be all you need for Classification Rishit Dagli 36 7 0 03 Apr 2023