ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases

19 March 2021

Papers citing "ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases"

50 / 399 papers shown

Title
Distilling Efficient Vision Transformers from CNNs for Semantic Segmentation Xueye Zheng Yunhao Luo Pengyuan Zhou Lin Wang 32 12 0 11 Oct 2023
Plug n' Play: Channel Shuffle Module for Enhancing Tiny Vision Transformers Xuwei Xu Sen Wang Yudong Chen Jiajun Liu ViT 21 1 0 09 Oct 2023
Enhancing Representations through Heterogeneous Self-Supervised Learning Zhongyu Li Bo-Wen Yin Yongxiang Liu Li Liu Ming-Ming Cheng SSL 23 2 0 08 Oct 2023
A Primer on Bayesian Neural Networks: Review and Debates Federico Danieli Konstantinos Pitas M. Vladimirova Vincent Fortuin BDL AAML 56 18 0 28 Sep 2023
Multi-Dimensional Hyena for Spatial Inductive Bias Itamar Zimerman Lior Wolf ViT 25 4 0 24 Sep 2023
RBFormer: Improve Adversarial Robustness of Transformer by Robust Bias Hao Cheng Jinhao Duan Hui Li Lyutianyang Zhang Jiahang Cao Ping Wang Jize Zhang Kaidi Xu Renjing Xu AAML 32 3 0 23 Sep 2023
NoisyNN: Exploring the Influence of Information Entropy Change in Learning Systems Xiao-Xing Yu Zhe Huang Yao Xue Lu Zhang Li Wang Tianming Liu Dajiang Zhu 18 6 0 19 Sep 2023
SCT: A Simple Baseline for Parameter-Efficient Fine-Tuning via Salient Channels Henry Hengyuan Zhao Pichao Wang Yuyang Zhao Hao Luo F. Wang Mike Zheng Shou ViT 34 14 0 15 Sep 2023
Keep It SimPool: Who Said Supervised Transformers Suffer from Attention Deficit? Bill Psomas Ioannis Kakogeorgiou Konstantinos Karantzalos Yannis Avrithis ViT 25 8 0 13 Sep 2023
Divergences in Color Perception between Deep Neural Networks and Humans E. Nadler Elise Darragh-Ford Bhargav Srinivasa Desikan Christian Conaway Mark Chu Tasker Hull Douglas Guilbeault 26 7 0 11 Sep 2023
DAT++: Spatially Dynamic Vision Transformer with Deformable Attention Zhuofan Xia Xuran Pan Shiji Song Li Erran Li Gao Huang ViT 21 24 0 04 Sep 2023
Computation-efficient Deep Learning for Computer Vision: A Survey Yulin Wang Yizeng Han Chaofei Wang Shiji Song Qi Tian Gao Huang VLM 31 20 0 27 Aug 2023
SPANet: Frequency-balancing Token Mixer using Spectral Pooling Aggregation Modulation Guhnoo Yun J. Yoo Kijung Kim Jeongho Lee Dong Hwan Kim MoE 20 8 0 22 Aug 2023
Exemplar-Free Continual Transformer with Convolutions Anurag Roy Vinay K. Verma Sravan Voonna Kripabandhu Ghosh Saptarshi Ghosh Abir Das CLL BDL 15 10 0 22 Aug 2023
Patch Is Not All You Need Chang-bo Li Jie M. Zhang Yang Wei Zhilong Ji Jinfeng Bai Shiguang Shan ViT 46 1 0 21 Aug 2023
Efficient Representation Learning for Healthcare with Cross-Architectural Self-Supervision P. Singh Jacopo Cirrone OOD SSL 27 2 0 19 Aug 2023
SCSC: Spatial Cross-scale Convolution Module to Strengthen both CNNs and Transformers Xijun Wang Xiaojie Chu Chunrui Han Xiangyu Zhang ViT 18 1 0 14 Aug 2023
SSTFormer: Bridging Spiking Neural Network and Memory Support Transformer for Frame-Event based Recognition Xiao Wang Zong-Yao Wu Yao Rong Lin Zhu Bowei Jiang Jin Tang Yonghong Tian ViT 71 15 0 08 Aug 2023
Distributionally Robust Classification on a Data Budget Ben Feuer Ameya Joshi Minh Pham C. Hegde OOD 32 2 0 07 Aug 2023
Heterogeneous Forgetting Compensation for Class-Incremental Learning Jiahua Dong Wenqi Liang Yang Cong Gan Sun CLL 26 19 0 07 Aug 2023
ViT2EEG: Leveraging Hybrid Pretrained Vision Transformers for EEG Data Ruiqi Yang Eric Modesitt ViT 23 12 0 01 Aug 2023
Robust face anti-spoofing framework with Convolutional Vision Transformer Yunseung Lee Youngjun Kwak Jinho Shin CVBM ViT 23 5 0 24 Jul 2023
A Holistic Assessment of the Reliability of Machine Learning Systems Anthony Corso David Karamadian Romeo Valentin Mary Cooper Mykel J. Kochenderfer 28 6 0 20 Jul 2023
DARTS: Double Attention Reference-based Transformer for Super-resolution Masoomeh Aslahishahri Jordan R. Ubbens Ian Stavness 25 3 0 17 Jul 2023
Transformers in Reinforcement Learning: A Survey Pranav Agarwal A. Rahman P. St-Charles Simon J. D. Prince Samira Ebrahimi Kahou OffRL 24 18 0 12 Jul 2023
End-To-End Prediction of Knee Osteoarthritis Progression With Multi-Modal Transformers E. Panfilov S. Saarakkala M. Nieminen A. Tiulpin 22 1 0 03 Jul 2023
DoseDiff: Distance-aware Diffusion Model for Dose Prediction in Radiotherapy Yiwen Zhang Chuanpu Li Liming Zhong Ze-xiang Chen Wei Yang Xuetao Wang DiffM MedIm 13 10 0 28 Jun 2023
Boosting Adversarial Transferability with Learnable Patch-wise Masks Xingxing Wei Shiji Zhao AAML 16 10 0 28 Jun 2023
TaCA: Upgrading Your Visual Foundation Model with Task-agnostic Compatible Adapter Binjie Zhang Yixiao Ge Xuyuan Xu Ying Shan Mike Zheng Shou 47 7 0 22 Jun 2023
Concurrent ischemic lesion age estimation and segmentation of CT brain using a Transformer-based network A. Marcus P. Bentley Daniel Rueckert MedIm 16 9 0 21 Jun 2023
NAR-Former V2: Rethinking Transformer for Universal Neural Network Representation Learning Yun Yi Haokui Zhang Rong Xiao Nan Wang Xiaoyu Wang GNN 24 2 0 19 Jun 2023
A Comprehensive Survey on Applications of Transformers for Deep Learning Tasks Saidul Islam Hanae Elmekki Ahmed Elsebai Jamal Bentahar Najat Drawel Gaith Rjoub Witold Pedrycz ViT MedIm 21 171 0 11 Jun 2023
2-D SSM: A General Spatial Layer for Visual Transformers Ethan Baron Itamar Zimerman Lior Wolf 28 14 0 11 Jun 2023
Connectional-Style-Guided Contextual Representation Learning for Brain Disease Diagnosis Gongshu Wang Ning Jiang Yunxiao Ma Tiantian Liu Duanduan Chen Jinglong Wu Guoqi Li Dong Liang Tianyi Yan MedIm 20 2 0 08 Jun 2023
Multi-Architecture Multi-Expert Diffusion Models Yunsung Lee Jin-Young Kim Hyojun Go Myeongho Jeong Shinhyeok Oh Seungtaek Choi DiffM 26 29 0 08 Jun 2023
Performance-optimized deep neural networks are evolving into worse models of inferotemporal visual cortex Drew Linsley I. F. Rodriguez Thomas Fel Michael Arcaro Saloni Sharma Margaret Livingstone Thomas Serre 35 18 0 06 Jun 2023
Are Large Kernels Better Teachers than Transformers for ConvNets? Tianjin Huang Lu Yin Zhenyu (Allen) Zhang Lijuan Shen Meng Fang Mykola Pechenizkiy Zhangyang Wang Shiwei Liu 30 13 0 30 May 2023
Introducing Competition to Boost the Transferability of Targeted Adversarial Examples through Clean Feature Mixup Junyoung Byun Myung-Joon Kwon Seungju Cho Yoonji Kim Changick Kim AAML 22 17 0 24 May 2023
How Deep Learning Sees the World: A Survey on Adversarial Attacks & Defenses Joana Cabral Costa Tiago Roxo Hugo Manuel Proença Pedro R. M. Inácio AAML 37 49 0 18 May 2023
Mimetic Initialization of Self-Attention Layers Asher Trockman J. Zico Kolter 28 30 0 16 May 2023
InternGPT: Solving Vision-Centric Tasks by Interacting with ChatGPT Beyond Language Zhaoyang Liu Yinan He Wenhai Wang Weiyun Wang Yi Wang ... Yali Wang Limin Wang Ping Luo Jifeng Dai Yu Qiao LRM MLLM 22 79 0 09 May 2023
BiRT: Bio-inspired Replay in Vision Transformers for Continual Learning Kishaan Jeeveswaran Prashant Bhat Bahram Zonooz Elahe Arani CLL 19 19 0 08 May 2023
What Do Self-Supervised Vision Transformers Learn? Namuk Park Wonjae Kim Byeongho Heo Taekyung Kim Sangdoo Yun SSL 67 76 1 01 May 2023
Deeply-Coupled Convolution-Transformer with Spatial-temporal Complementary Learning for Video-based Person Re-identification Xuehu Liu Chenyang Yu Pingping Zhang Huchuan Lu ViT 79 25 0 27 Apr 2023
Vision Conformer: Incorporating Convolutions into Vision Transformer Layers Brian Kenji Iwana Akihiro Kusuda ViT 43 2 0 27 Apr 2023
Preserving Locality in Vision Transformers for Class Incremental Learning Bowen Zheng Da-Wei Zhou Han-Jia Ye De-Chuan Zhan CLL 19 5 0 14 Apr 2023
SpectFormer: Frequency and Attention is what you need in a Vision Transformer Badri N. Patro Vinay P. Namboodiri Vijay Srinivas Agneeswaran ViT 29 47 0 13 Apr 2023
ViT-Calibrator: Decision Stream Calibration for Vision Transformer Lin Chen Zhijie Jia Tian Qiu Lechao Cheng Jie Lei Zunlei Feng Min-Gyoo Song 19 1 0 10 Apr 2023
MC-MLP:Multiple Coordinate Frames in all-MLP Architecture for Vision Zhimin Zhu Jianguo Zhao Tong Mu Yuliang Yang Mengyu Zhu 27 0 0 08 Apr 2023
PopulAtion Parameter Averaging (PAPA) Alexia Jolicoeur-Martineau Emy Gervais Kilian Fatras Yan Zhang Simon Lacoste-Julien MoMe 40 17 0 06 Apr 2023