ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases

19 March 2021

Papers citing "ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases"

50 / 399 papers shown

Title
Convolutional Neural Networks and Vision Transformers for Fashion MNIST Classification: A Literature Review Sonia Bbouzidi Ghazala Hcini Imen Jdey Fadoua Drira 23 4 0 05 Jun 2024
Automatic Channel Pruning for Multi-Head Attention Eunho Lee Youngbae Hwang ViT 40 1 0 31 May 2024
ViG: Linear-complexity Visual Sequence Learning with Gated Linear Attention Bencheng Liao Xinggang Wang Lianghui Zhu Qian Zhang Chang Huang 54 4 0 28 May 2024
Configuring Data Augmentations to Reduce Variance Shift in Positional Embedding of Vision Transformers Bum Jun Kim Sang Woo Kim ViT 41 1 0 23 May 2024
LookHere: Vision Transformers with Directed Attention Generalize and Extrapolate A. Fuller Daniel G. Kyrollos Yousef Yassin James R. Green 46 2 0 22 May 2024
Improving Transferable Targeted Adversarial Attack via Normalized Logit Calibration and Truncated Feature Mixing Juanjuan Weng Zhiming Luo Shaozi Li AAML 23 0 0 10 May 2024
Exploring Frequencies via Feature Mixing and Meta-Learning for Improving Adversarial Transferability Juanjuan Weng Zhiming Luo Shaozi Li AAML 24 1 0 06 May 2024
Cross-Temporal Spectrogram Autoencoder (CTSAE): Unsupervised Dimensionality Reduction for Clustering Gravitational Wave Glitches Yi Li Yunan Wu Aggelos K. Katsaggelos 21 1 0 23 Apr 2024
An Experimental Study on Exploring Strong Lightweight Vision Transformers via Masked Image Modeling Pre-Training Jin Gao Shubo Lin Shaoru Wang Yutong Kou Zeming Li Liang Li Congxuan Zhang Xiaoqin Zhang Yizheng Wang Weiming Hu 41 1 0 18 Apr 2024
GhostNetV3: Exploring the Training Strategies for Compact Models Zhenhua Liu Zhiwei Hao Kai Han Yehui Tang Yunhe Wang 24 16 0 17 Apr 2024
TSLANet: Rethinking Transformers for Time Series Representation Learning Emadeldeen Eldele Mohamed Ragab Zhenghua Chen Min-man Wu Xiaoli Li AI4TS AIFin 36 35 0 12 Apr 2024
Lightweight Deep Learning for Resource-Constrained Environments: A Survey Hou-I Liu Marco Galindo Hongxia Xie Lai-Kuan Wong Hong-Han Shuai Yung-Hui Li Wen-Huang Cheng 55 48 0 08 Apr 2024
Rethinking Self-training for Semi-supervised Landmark Detection: A Selection-free Approach Haibo Jin Haoxuan Che Hao Chen 45 0 0 06 Apr 2024
Performance of computer vision algorithms for fine-grained classification using crowdsourced insect images Rita Pucci Vincent J. Kalkman Dan Stowell 16 2 0 04 Apr 2024
ViTamin: Designing Scalable Vision Models in the Vision-Language Era Jienneg Chen Qihang Yu Xiaohui Shen Alan L. Yuille Liang-Chieh Chen 3DV VLM 33 24 0 02 Apr 2024
Structured Initialization for Attention in Vision Transformers Jianqiao Zheng Xueqian Li Simon Lucey ViT 21 1 0 01 Apr 2024
Action Detection via an Image Diffusion Process Lin Geng Foo Tianjiao Li Hossein Rahmani Jun Liu 22 4 0 01 Apr 2024
Seeing the Unseen: A Frequency Prompt Guided Transformer for Image Restoration Shihao Zhou Jinshan Pan Jinglei Shi Duosheng Chen Lishen Qu Jufeng Yang VLM 23 3 0 30 Mar 2024
Look-Around Before You Leap: High-Frequency Injected Transformer for Image Restoration Shihao Zhou Duosheng Chen Jinshan Pan Jufeng Yang 35 2 0 30 Mar 2024
Transformers-based architectures for stroke segmentation: A review Yalda Zafari-Ghadim Essam A. Rashed M. Mabrok MedIm 20 1 0 27 Mar 2024
Heracles: A Hybrid SSM-Transformer Model for High-Resolution Image and Time-Series Analysis Badri N. Patro Suhas Ranganath Vinay P. Namboodiri Vijay Srinivas Agneeswaran 43 2 0 26 Mar 2024
PlainMamba: Improving Non-Hierarchical Mamba in Visual Recognition Chenhongyi Yang Zehui Chen Miguel Espinosa Linus Ericsson Zhenyu Wang Jiaming Liu Elliot J. Crowley Mamba 36 86 0 26 Mar 2024
Enhancing Visual Continual Learning with Language-Guided Supervision Bolin Ni Hongbo Zhao Chenghao Zhang Ke Hu Gaofeng Meng Zhaoxiang Zhang Shiming Xiang CLL VLM 37 3 0 24 Mar 2024
LiFT: A Surprisingly Simple Lightweight Feature Transform for Dense ViT Descriptors Saksham Suri Matthew Walmer Kamal Gupta Abhinav Shrivastava 35 4 0 21 Mar 2024
HIRI-ViT: Scaling Vision Transformer with High Resolution Inputs Ting Yao Yehao Li Yingwei Pan Tao Mei ViT 25 15 0 18 Mar 2024
depyf: Open the Opaque Box of PyTorch Compiler for Machine Learning Researchers Kaichao You Runsheng Bai Meng Cao Jianmin Wang Ion Stoica Mingsheng Long VLM 33 0 0 14 Mar 2024
Learning without Exact Guidance: Updating Large-scale High-resolution Land Cover Maps from Low-resolution Historical Labels Zhuo Li Wei He Jiepan Li Fangxiao Lu Hongyan Zhang 23 12 0 05 Mar 2024
Rethinking Inductive Biases for Surface Normal Estimation Gwangbin Bae Andrew J. Davison 51 41 0 01 Mar 2024
Exploring the Synergies of Hybrid CNNs and ViTs Architectures for Computer Vision: A survey Haruna Yunusa Shiyin Qin Abdulrahman Hamman Adama Chukkol Abdulganiyu Abdu Yusuf Isah Bello A. Lawan ViT 30 13 0 05 Feb 2024
Learning from Teaching Regularization: Generalizable Correlations Should be Easy to Imitate Can Jin Tong Che Hongwu Peng Yiyuan Li Dimitris N. Metaxas Marco Pavone 44 43 0 05 Feb 2024
Graph-Mamba: Towards Long-Range Graph Sequence Modeling with Selective State Spaces Chloe X. Wang Oleksii Tsepa Jun Ma Bo Wang Mamba 25 86 0 01 Feb 2024
LDCA: Local Descriptors with Contextual Augmentation for Few-Shot Learning Maofa Wang Bingchen Yan 18 0 0 24 Jan 2024
Facing the Elephant in the Room: Visual Prompt Tuning or Full Finetuning? Cheng Han Qifan Wang Yiming Cui Wenguan Wang Lifu Huang Siyuan Qi Dongfang Liu VLM 44 19 0 23 Jan 2024
Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model Lianghui Zhu Bencheng Liao Qian Zhang Xinlong Wang Wenyu Liu Xinggang Wang Mamba 47 708 0 17 Jan 2024
Anatomy of Neural Language Models Majd Saleh Stéphane Paquelet MedIm 17 1 0 08 Jan 2024
SpecFormer: Guarding Vision Transformer Robustness via Maximum Singular Value Penalization Xixu Hu Runkai Zheng Jindong Wang Cheuk Hang Leung Qi Wu Xing Xie 27 1 0 02 Jan 2024
Cached Transformers: Improving Transformers with Differentiable Memory Cache Zhaoyang Zhang Wenqi Shao Yixiao Ge Xiaogang Wang Jinwei Gu Ping Luo 16 2 0 20 Dec 2023
Weighted Ensemble Models Are Strong Continual Learners Imad Eddine Marouf Subhankar Roy Enzo Tartaglione Stéphane Lathuilière CLL 27 16 0 14 Dec 2023
The Counterattack of CNNs in Self-Supervised Learning: Larger Kernel Size might be All You Need Tianjin Huang Tianlong Chen Zhangyang Wang Shiwei Liu 32 1 0 09 Dec 2023
MIMIR: Masked Image Modeling for Mutual Information-based Adversarial Robustness Xiaoyun Xu Shujian Yu Jingzheng Wu S. Picek AAML 35 0 0 08 Dec 2023
Rethinking Urban Mobility Prediction: A Super-Multivariate Time Series Forecasting Approach Jinguo Cheng Ke Li Yuxuan Liang Lijun Sun Junchi Yan Yuankai Wu AI4TS 27 2 0 04 Dec 2023
Corner-to-Center Long-range Context Model for Efficient Learned Image Compression Yang Sui Ding Ding Xiang Pan Xiaozhong Xu Shan Liu Bo Yuan Zhenzhong Chen 15 5 0 29 Nov 2023
Navigating Scaling Laws: Compute Optimality in Adaptive Model Training Sotiris Anagnostidis Gregor Bachmann Imanol Schlag Thomas Hofmann 33 2 0 06 Nov 2023
GTP-ViT: Efficient Vision Transformers via Graph-based Token Propagation Xuwei Xu Sen Wang Yudong Chen Yanping Zheng Zhewei Wei Jiajun Liu ViT 22 8 0 06 Nov 2023
Task Arithmetic with LoRA for Continual Learning Rajas Chitale Ankit Vaidya Aditya Kane Archana Ghotkar 32 13 0 04 Nov 2023
Scattering Vision Transformer: Spectral Mixing Matters Badri N. Patro Vijay Srinivas Agneeswaran 29 14 0 02 Nov 2023
Improving Robustness and Reliability in Medical Image Classification with Latent-Guided Diffusion and Nested-Ensembles Xing Shen Hengguan Huang Brennan Nichyporuk Tal Arbel MedIm 38 4 0 24 Oct 2023
The Efficacy of Transformer-based Adversarial Attacks in Security Domains Kunyang Li Kyle Domico Jean-Charles Noirot Ferrand Patrick D. McDaniel AAML 26 0 0 17 Oct 2023
Domain Generalization Using Large Pretrained Models with Mixture-of-Adapters Gyuseong Lee Wooseok Jang Jin Hyeon Kim Jaewoo Jung Seungryong Kim MoE OOD 17 2 0 17 Oct 2023
Accelerating Vision Transformers Based on Heterogeneous Attention Patterns Deli Yu Teng Xi Jianwei Li Baopu Li Gang Zhang Haocheng Feng Junyu Han Jingtuo Liu Errui Ding Jingdong Wang ViT 28 0 0 11 Oct 2023