ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases

19 March 2021

Papers citing "ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases"

50 / 399 papers shown

Title
Wave-ViT: Unifying Wavelet and Transformers for Visual Representation Learning Ting Yao Yingwei Pan Yehao Li Chong-Wah Ngo Tao Mei ViT 148 137 0 11 Jul 2022
Dual Vision Transformer Ting Yao Yehao Li Yingwei Pan Yu Wang Xiaoping Zhang Tao Mei ViT 141 75 0 11 Jul 2022
Vision Transformers: State of the Art and Research Challenges Bo-Kai Ruan Hong-Han Shuai Wen-Huang Cheng ViT 24 17 0 07 Jul 2022
Softmax-free Linear Transformers Jiachen Lu Junge Zhang Xiatian Zhu Jianfeng Feng Tao Xiang Li Zhang ViT 11 7 0 05 Jul 2022
The Lighter The Better: Rethinking Transformers in Medical Image Segmentation Through Adaptive Pruning Xian Lin Li Yu Kwang-Ting Cheng Zengqiang Yan ViT MedIm 22 31 0 29 Jun 2022
Agreement-on-the-Line: Predicting the Performance of Neural Networks under Distribution Shift Christina Baek Yiding Jiang Aditi Raghunathan Zico Kolter 24 79 0 27 Jun 2022
EdgeNeXt: Efficiently Amalgamated CNN-Transformer Architecture for Mobile Vision Applications Muhammad Maaz Abdelrahman M. Shaker Hisham Cholakkal Salman Khan Syed Waqas Zamir Rao Muhammad Anwer F. Khan ViT 27 184 0 21 Jun 2022
EATFormer: Improving Vision Transformer Inspired by Evolutionary Algorithm Jiangning Zhang Xiangtai Li Yabiao Wang Chengjie Wang Yibo Yang Yong Liu Dacheng Tao ViT 34 32 0 19 Jun 2022
Video Capsule Endoscopy Classification using Focal Modulation Guided Convolutional Neural Network Abhishek Srivastava Nikhil Kumar Tomar Ulas Bagci Debesh Jha MedIm 16 15 0 16 Jun 2022
Online Segmentation of LiDAR Sequences: Dataset and Algorithm Romain Loiseau Mathieu Aubry Loïc Landrieu 3DPC 19 15 0 16 Jun 2022
SP-ViT: Learning 2D Spatial Priors for Vision Transformers Yuxuan Zhou Wangmeng Xiang Chuan Li Biao Wang Xihan Wei Lei Zhang M. Keuper Xia Hua ViT 31 15 0 15 Jun 2022
Peripheral Vision Transformer Juhong Min Yucheng Zhao Chong Luo Minsu Cho ViT MDE 29 30 0 14 Jun 2022
SeATrans: Learning Segmentation-Assisted diagnosis model via Transformer Junde Wu Huihui Fang Fangxin Shang Dalu Yang Zhao-Yang Wang Jing Gao Yehui Yang Yanwu Xu MedIm ViT 17 19 0 12 Jun 2022
GAMR: A Guided Attention Model for (visual) Reasoning Mohit Vaishnav Thomas Serre LRM 19 16 0 10 Jun 2022
Spatial Entropy as an Inductive Bias for Vision Transformers E. Peruzzo E. Sangineto Yahui Liu Marco De Nadai Wei Bi Bruno Lepri N. Sebe ViT MDE 31 1 0 09 Jun 2022
CASS: Cross Architectural Self-Supervision for Medical Image Analysis Pranav Singh E. Sizikova Jacopo Cirrone OOD 52 8 0 08 Jun 2022
MobileOne: An Improved One millisecond Mobile Backbone Pavan Kumar Anasosalu Vasu J. Gabriel Jeff J. Zhu Oncel Tuzel Anurag Ranjan 25 154 0 08 Jun 2022
Recent Advances for Quantum Neural Networks in Generative Learning Jinkai Tian Xiaoyun Sun Yuxuan Du Shanshan Zhao Qing Liu ... Xingyao Wu Min-hsiu Hsieh Tongliang Liu Wen-Bin Yang Dacheng Tao AI4CE 24 81 0 07 Jun 2022
Separable Self-attention for Mobile Vision Transformers Sachin Mehta Mohammad Rastegari ViT MQ 18 251 0 06 Jun 2022
Which models are innately best at uncertainty estimation? Ido Galil Mohammed Dabbah Ran El-Yaniv UQCV 34 5 0 05 Jun 2022
Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectives Jun Li Junyu Chen Yucheng Tang Ce Wang Bennett A. Landman S. K. Zhou ViT OOD MedIm 21 20 0 02 Jun 2022
Surface Analysis with Vision Transformers Simon Dahan Logan Z. J. Williams Abdulah Fawaz Daniel Rueckert E. C. Robinson ViT MedIm 29 2 0 31 May 2022
Green Hierarchical Vision Transformer for Masked Image Modeling Lang Huang Shan You Mingkai Zheng Fei Wang Chao Qian T. Yamasaki 27 68 0 26 May 2022
MoCoViT: Mobile Convolutional Vision Transformer Hailong Ma Xin Xia Xing Wang Xuefeng Xiao Jiashi Li Min Zheng ViT 34 18 0 25 May 2022
Recipe for a General, Powerful, Scalable Graph Transformer Ladislav Rampášek Mikhail Galkin Vijay Prakash Dwivedi A. Luu Guy Wolf Dominique Beaini 57 515 0 25 May 2022
Dynamic Query Selection for Fast Visual Perceiver Corentin Dancette Matthieu Cord 28 1 0 22 May 2022
Scalable and Efficient Training of Large Convolutional Neural Networks with Differential Privacy Zhiqi Bu J. Mao Shiyun Xu 131 47 0 21 May 2022
BabyNet: Residual Transformer Module for Birth Weight Prediction on Fetal Ultrasound Video Szymon Płotka Michal K. Grzeszczyk R. Brawura-Biskupski-Samaha P. Gutaj M. Lipa Tomasz Trzciñski Arkadiusz Sitek 3DH MedIm 11 17 0 19 May 2022
ImageSig: A signature transform for ultra-lightweight image recognition Mohamed Ramzy Ibrahim Terry Lyons VLM 19 7 0 13 May 2022
A Continual Deepfake Detection Benchmark: Dataset, Methods, and Essentials Chuqiao Li Zhiwu Huang D. Paudel Yabin Wang Mohamad Shahbazi Xiaopeng Hong Luc Van Gool 20 48 0 11 May 2022
ConvMAE: Masked Convolution Meets Masked Autoencoders Peng Gao Teli Ma Hongsheng Li Ziyi Lin Jifeng Dai Yu Qiao ViT 19 121 0 08 May 2022
DearKD: Data-Efficient Early Knowledge Distillation for Vision Transformers Xianing Chen Qiong Cao Yujie Zhong Jing Zhang Shenghua Gao Dacheng Tao ViT 32 76 0 27 Apr 2022
Masked Spectrogram Prediction For Self-Supervised Audio Pre-Training Dading Chong Helin Wang Peilin Zhou Qingcheng Zeng 39 65 0 27 Apr 2022
A survey on attention mechanisms for medical applications: are we moving towards better algorithms? Tiago Gonçalves Isabel Rio-Torto Luís F. Teixeira J. S. Cardoso OOD MedIm 24 36 0 26 Apr 2022
Transformation Invariant Cancerous Tissue Classification Using Spatially Transformed DenseNet Omar Mahdi Ali Bou Nassif MedIm 9 2 0 23 Apr 2022
Visual Attention Emerges from Recurrent Sparse Reconstruction Baifeng Shi Ya-heng Song Neel Joshi Trevor Darrell Xin Wang 3DH 14 6 0 23 Apr 2022
DeiT III: Revenge of the ViT Hugo Touvron Matthieu Cord Hervé Jégou ViT 42 389 0 14 Apr 2022
3D Shuffle-Mixer: An Efficient Context-Aware Vision Learner of Transformer-MLP Paradigm for Dense Prediction in Medical Volume Jianye Pang Cheng Jiang Yihao Chen Jianbo Chang M. Feng Renzhi Wang Jianhua Yao ViT MedIm 28 11 0 14 Apr 2022
Vision Transformer Equipped with Neural Resizer on Facial Expression Recognition Task Hyeonbin Hwang Soyeon Kim Wei-Jin Park Jiho Seo Kyungtae Ko Hyeon Yeo ViT 39 9 0 05 Apr 2022
MaxViT: Multi-Axis Vision Transformer Zhengzhong Tu Hossein Talebi Han Zhang Feng Yang P. Milanfar A. Bovik Yinxiao Li ViT 53 636 0 04 Apr 2022
Improving Vision Transformers by Revisiting High-frequency Components Jiawang Bai Liuliang Yuan Shutao Xia Shuicheng Yan Zhifeng Li Wei Liu ViT 14 90 0 03 Apr 2022
Surface Vision Transformers: Attention-Based Modelling applied to Cortical Analysis Simon Dahan Abdulah Fawaz Logan Z. J. Williams Chunhui Yang Timothy S. Coalson M. Glasser A. Edwards Daniel Rueckert E. C. Robinson MedIm ViT 40 20 0 30 Mar 2022
Affine Medical Image Registration with Coarse-to-Fine Vision Transformer Tony C. W. Mok Albert C. S. Chung ViT MedIm 34 62 0 29 Mar 2022
Core Risk Minimization using Salient ImageNet Sahil Singla Mazda Moayeri S. Feizi 30 14 0 28 Mar 2022
CodedVTR: Codebook-based Sparse Voxel Transformer with Geometric Guidance Tianchen Zhao Niansong Zhang Xuefei Ning He-Nan Wang Li Yi Yu Wang 3DPC ViT 22 8 0 18 Mar 2022
Three things everyone should know about Vision Transformers Hugo Touvron Matthieu Cord Alaaeldin El-Nouby Jakob Verbeek Hervé Jégou ViT 21 119 0 18 Mar 2022
InvPT: Inverted Pyramid Multi-task Transformer for Dense Scene Understanding Hanrong Ye Dan Xu ViT 21 84 0 15 Mar 2022
Enriched CNN-Transformer Feature Aggregation Networks for Super-Resolution Jinsu Yoo Taehoon Kim Sihaeng Lee Seunghyeon Kim H. Lee Tae Hyun Kim SupR ViT 31 51 0 15 Mar 2022
ChiTransformer:Towards Reliable Stereo from Cues Qing Su Shihao Ji MDE ViT 18 12 0 09 Mar 2022
ParC-Net: Position Aware Circular Convolution with Merits from ConvNets and Transformer Haokui Zhang Wenze Hu Xiaoyu Wang ViT 41 59 0 08 Mar 2022