ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases

19 March 2021

Papers citing "ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases"

50 / 399 papers shown

Title
Towards an Effective and Efficient Transformer for Rain-by-snow Weather Removal Tao Gao Yuanbo Wen Kaihao Zhang Peng Cheng Ting Chen ViT 33 5 0 06 Apr 2023
What Affects Learned Equivariance in Deep Image Recognition Models? Robert-Jan Bruintjes Tomasz Motyka J. C. V. Gemert 20 7 0 05 Apr 2023
Astroformer: More Data Might not be all you need for Classification Rishit Dagli 28 7 0 03 Apr 2023
Transferable Adversarial Attacks on Vision Transformers with Token Gradient Regularization Jianping Zhang Yizhan Huang Weibin Wu Michael R. Lyu AAML ViT 18 49 0 28 Mar 2023
Data Augmentation for Environmental Sound Classification Using Diffusion Probabilistic Model with Top-k Selection Discriminator Yunhao Chen Yunjie Zhu Zihui Yan Jian Shen Zhen Ren Yifan Huang DiffM 39 8 0 27 Mar 2023
Sector Patch Embedding: An Embedding Module Conforming to The Distortion Pattern of Fisheye Image Dian Yang Jiadong Tang Yu Gao Yi Yang M. Fu 23 1 0 26 Mar 2023
FastViT: A Fast Hybrid Vision Transformer using Structural Reparameterization Pavan Kumar Anasosalu Vasu J. Gabriel Jeff J. Zhu Oncel Tuzel Anurag Ranjan ViT 37 153 0 24 Mar 2023
Top-Down Visual Attention from Analysis by Synthesis Baifeng Shi Trevor Darrell Xin Eric Wang 25 28 0 23 Mar 2023
Machine Learning for Brain Disorders: Transformers and Visual Transformers Robin Courant Maika Edberg Nicolas Dufour Vicky Kalogeiton MedIm ViT 32 1 0 21 Mar 2023
Large AI Models in Health Informatics: Applications, Challenges, and the Future Jianing Qiu Lin Li Jiankai Sun Jiachuan Peng Peilun Shi ... Bo Xiao Wu Yuan Ningli Wang Dong Xu Benny Lo AI4MH LM&MA 42 127 0 21 Mar 2023
FullFormer: Generating Shapes Inside Shapes Tejaswini Medi Jawad Tayyub M. Sarmad Frank Lindseth M. Keuper 35 0 0 20 Mar 2023
Robustifying Token Attention for Vision Transformers Yong Guo David Stutz Bernt Schiele ViT 21 24 0 20 Mar 2023
Revisit Parameter-Efficient Transfer Learning: A Two-Stage Paradigm Hengyuan Zhao Hao Luo Yuyang Zhao Pichao Wang F. Wang Mike Zheng Shou 26 5 0 14 Mar 2023
HyT-NAS: Hybrid Transformers Neural Architecture Search for Edge Devices Lotfi Abdelkrim Mecharbat Hadjer Benmeziane Hamza Ouarnoughi Smail Niar ViT 35 4 0 08 Mar 2023
Self-attention in Vision Transformers Performs Perceptual Grouping, Not Attention Paria Mehrani John K. Tsotsos 25 24 0 02 Mar 2023
Diffusing Graph Attention Daniel Glickman Eran Yahav GNN 47 3 0 01 Mar 2023
Structured Pruning for Deep Convolutional Neural Networks: A survey Yang He Lingao Xiao 3DPC 30 117 0 01 Mar 2023
Extracting Motion and Appearance via Inter-Frame Attention for Efficient Video Frame Interpolation Guozhen Zhang Yuhan Zhu Hongya Wang Youxin Chen Gangshan Wu Limin Wang 71 84 0 01 Mar 2023
Language Models are Few-shot Learners for Prognostic Prediction Zekai Chen Mariann Micsinai Balan Kevin Brown LM&MA 31 88 0 24 Feb 2023
A Convolutional Vision Transformer for Semantic Segmentation of Side-Scan Sonar Data Hayat Rajani N. Gracias Rafael García ViT 27 12 0 24 Feb 2023
Device Tuning for Multi-Task Large Model Penghao Jiang Xuanchen Hou Y. Zhou 26 0 0 21 Feb 2023
CECT: Controllable Ensemble CNN and Transformer for COVID-19 Image Classification Zhao Liu Leizhao Shen ViT 29 7 0 05 Feb 2023
SCCAM: Supervised Contrastive Convolutional Attention Mechanism for Ante-hoc Interpretable Fault Diagnosis with Limited Fault Samples Mengxuan Li Peng Peng Jingxin Zhang Hongwei Wang Nong Sang 22 17 0 03 Feb 2023
Multiple Thinking Achieving Meta-Ability Decoupling for Object Navigation Ronghao Dang Lu Chen Liuyi Wang Zongtao He Chengju Liu Qi Chen LRM 21 8 0 03 Feb 2023
Dual PatchNorm Manoj Kumar Mostafa Dehghani N. Houlsby UQCV ViT 29 11 0 02 Feb 2023
A Survey on Efficient Training of Transformers Bohan Zhuang Jing Liu Zizheng Pan Haoyu He Yuetian Weng Chunhua Shen 31 47 0 02 Feb 2023
Longformer: Longitudinal Transformer for Alzheimer's Disease Classification with Structural MRIs Qiu-hui Chen Yi Hong MedIm 22 11 0 02 Feb 2023
Cross-Architectural Positive Pairs improve the effectiveness of Self-Supervised Learning P. Singh Jacopo Cirrone SSL 40 0 0 27 Jan 2023
On the Connection Between MPNN and Graph Transformer Chen Cai Truong Son-Hy Rose Yu Yusu Wang 33 51 0 27 Jan 2023
Out of Distribution Performance of State of Art Vision Model Salman Rahman W. Lee 37 2 0 25 Jan 2023
HRVQA: A Visual Question Answering Benchmark for High-Resolution Aerial Images Kun Li G. Vosselman M. Yang 23 5 0 23 Jan 2023
Learning to Exploit Temporal Structure for Biomedical Vision-Language Processing Shruthi Bannur Stephanie L. Hyland Qianchu Liu Fernando Pérez-García Maximilian Ilse ... Maria T. A. Wetscherek M. Lungren A. Nori Javier Alvarez-Valle Ozan Oktay 34 110 0 11 Jan 2023
Advances in Medical Image Analysis with Vision Transformers: A Comprehensive Review Reza Azad A. Kazerouni Moein Heidari Ehsan Khodapanah Aghdam Amir Molaei Yiwei Jia Abin Jose Rijo Roy Dorit Merhof MedIm ViT 32 161 0 09 Jan 2023
Exploring Vision Transformers as Diffusion Learners He Cao Jianan Wang Tianhe Ren Xianbiao Qi Yihao Chen Yuan Yao L. Zhang 44 10 0 28 Dec 2022
SLGTformer: An Attention-Based Approach to Sign Language Recognition Neil Song Yu Xiang SLR 22 0 0 21 Dec 2022
Medical Diagnosis with Large Scale Multimodal Transformers: Leveraging Diverse Data for More Accurate Diagnosis Firas Khader Gustav Mueller-Franzes Tian Wang T. Han Soroosh Tayebi Arasteh ... Keno Bressem Christiane Kuhl S. Nebelung Jakob Nikolas Kather Daniel Truhn 11 6 0 18 Dec 2022
GPViT: A High Resolution Non-Hierarchical Vision Transformer with Group Propagation Chenhongyi Yang Jiarui Xu Shalini De Mello Elliot J. Crowley Xinyu Wang ViT 38 21 0 13 Dec 2022
What do Vision Transformers Learn? A Visual Exploration Amin Ghiasi Hamid Kazemi Eitan Borgnia Steven Reich Manli Shu Micah Goldblum A. Wilson Tom Goldstein ViT 26 60 0 13 Dec 2022
OAMixer: Object-aware Mixing Layer for Vision Transformers H. Kang Sangwoo Mo Jinwoo Shin VLM 39 4 0 13 Dec 2022
Vision Transformer with Attentive Pooling for Robust Facial Expression Recognition Fanglei Xue Qiangchang Wang Zichang Tan Zhongsong Ma G. Guo ViT 35 67 0 11 Dec 2022
Position Embedding Needs an Independent Layer Normalization Runyi Yu Zhennan Wang Yinhuai Wang Kehan Li Yian Zhao Jian Zhang Guoli Song Jie Chen 31 1 0 10 Dec 2022
Analysis of Deep Learning Architectures and Efficacy of Detecting Forest Fires Ryan Marinelli 23 0 0 08 Dec 2022
ResFormer: Scaling ViTs with Multi-Resolution Training Rui Tian Zuxuan Wu Qiuju Dai Hang-Rui Hu Yu Qiao Yu-Gang Jiang ViT 19 32 0 01 Dec 2022
Lightweight Structure-Aware Attention for Visual Understanding Heeseung Kwon F. M. Castro M. Marín-Jiménez N. Guil Alahari Karteek 28 2 0 29 Nov 2022
Adaptive Attention Link-based Regularization for Vision Transformers Heegon Jin Jongwon Choi ViT 14 0 0 25 Nov 2022
Spatial Mixture-of-Experts Nikoli Dryden Torsten Hoefler MoE 34 9 0 24 Nov 2022
Vision Transformers in Medical Imaging: A Review Emerald U. Henry Onyeka Emebob C. Omonhinmin ViT MedIm 24 34 0 18 Nov 2022
Fcaformer: Forward Cross Attention in Hybrid Vision Transformer Haokui Zhang Wenze Hu Xiaoyu Wang ViT 19 8 0 14 Nov 2022
Token Transformer: Can class token help window-based transformer build better long-range interactions? Jia-ju Mao Yuan Chang Xuesong Yin 28 0 0 11 Nov 2022
Unifying Flow, Stereo and Depth Estimation Haofei Xu Jing Zhang Jianfei Cai Hamid Rezatofighi F. I. F. Richard Yu Dacheng Tao Andreas Geiger MDE 31 193 0 10 Nov 2022