v1v2 (latest)

CoAtNet: Marrying Convolution and Attention for All Data Sizes

9 June 2021

Mingxing Tan

Papers citing "CoAtNet: Marrying Convolution and Attention for All Data Sizes"

50 / 64 papers shown

Title
Swin DiT: Diffusion Transformer using Pseudo Shifted Windows Jiafu Wu Yabiao Wang Jian Li Jinlong Peng Yun Cao Chengjie Wang Jiangning Zhang 189 0 0 19 May 2025
DCS-ST for Classification of Breast Cancer Histopathology Images with Limited Annotations Liu Suxing Byungwon Min 241 0 0 06 May 2025
HGFormer: Topology-Aware Vision Transformer with HyperGraph Learning Hao Wang Shuo Zhang Biao Leng ViT 266 1 0 03 Apr 2025
vGamba: Attentive State Space Bottleneck for efficient Long-range Dependencies in Visual Recognition Yunusa Haruna A. Lawan Mamba 116 0 0 27 Mar 2025
E2ENet: Dynamic Sparse Feature Fusion for Accurate and Efficient 3D Medical Image Segmentation Boqian Wu Q. Xiao Shiwei Liu Lu Yin Mykola Pechenizkiy Decebal Constantin Mocanu M. V. Keulen Elena Mocanu MedIm 123 6 0 20 Feb 2025
VMamba: Visual State Space Model Yue Liu Yunjie Tian Yuzhong Zhao Hongtian Yu Lingxi Xie Yaowei Wang Qixiang Ye Jianbin Jiao Yunfan Liu Mamba 299 699 0 31 Dec 2024
Breaking the Low-Rank Dilemma of Linear Attention Qihang Fan Huaibo Huang Ran He 101 2 0 12 Nov 2024
Designing Concise ConvNets with Columnar Stages Ashish Kumar Jaesik Park MQ 102 0 0 05 Oct 2024
Accuracy Improvement of Cell Image Segmentation Using Feedback Former Hinako Mitsuoka Kazuhiro Hotta ViT MedIm 211 0 0 23 Aug 2024
iiANET: Inception Inspired Attention Hybrid Network for efficient Long-Range Dependency Haruna Yunusa Qin Shiyin Abdulrahman Hamman Adama Chukkol Isah Bello A. Lawan Isah Bello 87 4 0 10 Jul 2024
Enhancing Domain Adaptation through Prompt Gradient Alignment Hoang Phan Lam C. Tran Quyen Tran Trung Le 131 1 0 13 Jun 2024
The 3D-PC: a benchmark for visual perspective taking in humans and machines Drew Linsley Peisen Zhou A. Ashok Akash Nagaraj Gaurav Gaonkar Francis E Lewis Zygmunt Pizlo Thomas Serre 119 6 0 06 Jun 2024
Are queries and keys always relevant? A case study on Transformer wave functions Riccardo Rende Luciano Loris Viteritti 63 7 0 29 May 2024
A Survey on Vision-Language-Action Models for Embodied AI Yueen Ma Zixing Song Yuzheng Zhuang Jianye Hao Irwin King LM&Ro 300 54 0 23 May 2024
Tiny Models are the Computational Saver for Large Models Qingyuan Wang B. Cardiff Antoine Frappé Benoît Larras Deepu John 88 2 0 26 Mar 2024
Delving Deeper Into Astromorphic Transformers Md. Zesun Ahmed Mia Malyaban Bal Abhronil Sengupta 138 1 0 18 Dec 2023
SCHEME: Scalable Channel Mixer for Vision Transformers Deepak Sridhar Yunsheng Li Nuno Vasconcelos 125 0 0 01 Dec 2023
TransXNet: Learning Both Global and Local Dynamics with a Dual Dynamic Token Mixer for Visual Recognition Meng Lou Hong-Yu Zhou Sibei Yang Yizhou Yu Chuan Wu Yizhou Yu ViT 130 41 0 30 Oct 2023
Scaling Vision Transformers Xiaohua Zhai Alexander Kolesnikov N. Houlsby Lucas Beyer ViT 141 1,094 0 08 Jun 2021
LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inference Ben Graham Alaaeldin El-Nouby Hugo Touvron Pierre Stock Armand Joulin Hervé Jégou Matthijs Douze ViT 93 792 0 02 Apr 2021
EfficientNetV2: Smaller Models and Faster Training Mingxing Tan Quoc V. Le EgoV 127 2,720 0 01 Apr 2021
Going deeper with Image Transformers Hugo Touvron Matthieu Cord Alexandre Sablayrolles Gabriel Synnaeve Hervé Jégou ViT 160 1,021 0 31 Mar 2021
CvT: Introducing Convolutions to Vision Transformers Haiping Wu Bin Xiao Noel Codella Mengchen Liu Xiyang Dai Lu Yuan Lei Zhang ViT 154 1,917 0 29 Mar 2021
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows Ze Liu Yutong Lin Yue Cao Han Hu Yixuan Wei Zheng Zhang Stephen Lin B. Guo ViT 465 21,566 0 25 Mar 2021
Scaling Local Self-Attention for Parameter Efficient Visual Backbones Ashish Vaswani Prajit Ramachandran A. Srinivas Niki Parmar Blake A. Hechtman Jonathon Shlens 94 403 0 23 Mar 2021
DeepViT: Towards Deeper Vision Transformer Daquan Zhou Bingyi Kang Xiaojie Jin Linjie Yang Xiaochen Lian Zihang Jiang Qibin Hou Jiashi Feng ViT 102 523 0 22 Mar 2021
Incorporating Convolution Designs into Visual Transformers Kun Yuan Shaopeng Guo Ziwei Liu Aojun Zhou F. Yu Wei Wu ViT 111 480 0 22 Mar 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 535 3,734 0 24 Feb 2021
LambdaNetworks: Modeling Long-Range Interactions Without Attention Irwan Bello 338 181 0 17 Feb 2021
High-Performance Large-Scale Image Recognition Without Normalization Andrew Brock Soham De Samuel L. Smith Karen Simonyan VLM 285 523 0 11 Feb 2021
Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet Li-xin Yuan Yunpeng Chen Tao Wang Weihao Yu Yujun Shi Zihang Jiang Francis E. H. Tay Jiashi Feng Shuicheng Yan ViT 143 1,942 0 28 Jan 2021
Bottleneck Transformers for Visual Recognition A. Srinivas Nayeon Lee Niki Parmar Jonathon Shlens Pieter Abbeel Ashish Vaswani SLR 360 994 0 27 Jan 2021
Transformers in Vision: A Survey Salman Khan Muzammal Naseer Munawar Hayat Syed Waqas Zamir Fahad Shahbaz Khan M. Shah ViT 307 2,528 0 04 Jan 2021
Training data-efficient image transformers & distillation through attention Hugo Touvron Matthieu Cord Matthijs Douze Francisco Massa Alexandre Sablayrolles Hervé Jégou ViT 389 6,802 0 23 Dec 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 673 41,430 0 22 Oct 2020
Rethinking Attention with Performers K. Choromanski Valerii Likhosherstov David Dohan Xingyou Song Andreea Gane ... Afroz Mohiuddin Lukasz Kaiser David Belanger Lucy J. Colwell Adrian Weller 186 1,602 0 30 Sep 2020
Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention Angelos Katharopoulos Apoorv Vyas Nikolaos Pappas Franccois Fleuret 203 1,786 0 29 Jun 2020
Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing Zihang Dai Guokun Lai Yiming Yang Quoc V. Le 90 234 0 05 Jun 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 880 42,379 0 28 May 2020
A Data and Compute Efficient Design for Limited-Resources Deep Learning Mirgahney Mohamed Gabriele Cesa Taco S. Cohen Max Welling MedIm 70 18 0 21 Apr 2020
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 485 20,317 0 23 Oct 2019
RandAugment: Practical automated data augmentation with a reduced search space E. D. Cubuk Barret Zoph Jonathon Shlens Quoc V. Le MQ 258 3,502 0 30 Sep 2019
Transformer Dissection: A Unified Understanding of Transformer's Attention via the Lens of Kernel Yao-Hung Hubert Tsai Shaojie Bai M. Yamada Louis-Philippe Morency Ruslan Salakhutdinov 129 261 0 30 Aug 2019
Stand-Alone Self-Attention in Vision Models Prajit Ramachandran Niki Parmar Ashish Vaswani Irwan Bello Anselm Levskaya Jonathon Shlens VLM SLR ViT 104 1,216 0 13 Jun 2019
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks Mingxing Tan Quoc V. Le 3DV MedIm 161 18,179 0 28 May 2019
Attention Augmented Convolutional Networks Irwan Bello Barret Zoph Ashish Vaswani Jonathon Shlens Quoc V. Le 143 1,015 0 22 Apr 2019
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context Zihang Dai Zhilin Yang Yiming Yang J. Carbonell Quoc V. Le Ruslan Salakhutdinov VLM 260 3,745 0 09 Jan 2019
Efficient Attention: Attention with Linear Complexities Zhuoran Shen Mingyuan Zhang Haiyu Zhao Shuai Yi Hongsheng Li 95 533 0 04 Dec 2018
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 95,175 0 11 Oct 2018
MnasNet: Platform-Aware Neural Architecture Search for Mobile Mingxing Tan Bo Chen Ruoming Pang Vijay Vasudevan Mark Sandler Andrew G. Howard Quoc V. Le MQ 128 3,015 0 31 Jul 2018