DaViT: Dual Attention Vision Transformers

7 April 2022

Mingyu Ding

Bin Xiao

Noel Codella

Ping Luo

Jingdong Wang

Lu Yuan

ViT

ArXiv PDF HTML

Papers citing "DaViT: Dual Attention Vision Transformers"

28 / 128 papers shown

Title
A Dual-scale Lead-seperated Transformer With Lead-orthogonal Attention And Meta-information For Ecg Classification Heng Chang Guijin Wang Zhourui Xia Wenming Yang Li Sun MedIm 34 1 0 23 Nov 2022
N-Gram in Swin Transformers for Efficient Lightweight Image Super-Resolution Haram Choi Jeong-Sik Lee Jihoon Yang ViT 24 75 0 21 Nov 2022
WSC-Trans: A 3D network model for automatic multi-structural segmentation of temporal bone CT X. Hua Z. Du Hongjian Yu Ji-Xian Ma Fanjun Zheng Cheng Zhang Qiao-Ling Lu H Zhao MedIm 13 1 0 14 Nov 2022
Token Transformer: Can class token help window-based transformer build better long-range interactions? Jia-ju Mao Yuan Chang Xuesong Yin 31 0 0 11 Nov 2022
InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions Wenhai Wang Jifeng Dai Zhe Chen Zhenhang Huang Zhiqi Li ... Tong Lu Lewei Lu Hongsheng Li Xiaogang Wang Yu Qiao VLM 38 657 0 10 Nov 2022
Accumulated Trivial Attention Matters in Vision Transformers on Small Datasets Xiangyu Chen Qinghao Hu Kaidong Li Cuncong Zhong Guanghui Wang ViT 38 11 0 22 Oct 2022
How Does a Deep Learning Model Architecture Impact Its Privacy? A Comprehensive Study of Privacy Attacks on CNNs and Transformers Guangsheng Zhang B. Liu Huan Tian Tianqing Zhu Ming Ding Wanlei Zhou PILM MIACV 20 5 0 20 Oct 2022
S4ND: Modeling Images and Videos as Multidimensional Signals Using State Spaces Eric N. D. Nguyen Karan Goel Albert Gu Gordon W. Downs Preey Shah Tri Dao S. Baccus Christopher Ré VLM 22 39 0 12 Oct 2022
MAFormer: A Transformer Network with Multi-scale Attention Fusion for Visual Recognition Y. Wang H. Sun Xiaodi Wang Bin Zhang Chaonan Li Ying Xin Baochang Zhang Errui Ding Shumin Han ViT 31 9 0 31 Aug 2022
Transformer Vs. MLP-Mixer: Exponential Expressive Gap For NLP Problems D. Navon A. Bronstein MoE 38 0 0 17 Aug 2022
Unified Normalization for Accelerating and Stabilizing Transformers Qiming Yang Kai Zhang Chaoxiang Lan Zhi Yang Zheyang Li Wenming Tan Jun Xiao Shiliang Pu 15 8 0 02 Aug 2022
GIT: A Generative Image-to-text Transformer for Vision and Language Jianfeng Wang Zhengyuan Yang Xiaowei Hu Linjie Li Kevin Qinghong Lin Zhe Gan Zicheng Liu Ce Liu Lijuan Wang VLM 59 528 0 27 May 2022
RFNet-4D++: Joint Object Reconstruction and Flow Estimation from 4D Point Clouds with Cross-Attention Spatio-Temporal Features Tuan-Anh Vu D. Nguyen Binh-Son Hua Quang-Cuong Pham Sai-Kit Yeung 3DPC 55 4 0 30 Mar 2022
Focal Modulation Networks Jianwei Yang Chunyuan Li Xiyang Dai Lu Yuan Jianfeng Gao 3DPC 33 263 0 22 Mar 2022
Patches Are All You Need? Asher Trockman J. Zico Kolter ViT 225 402 0 24 Jan 2022
UniFormer: Unifying Convolution and Self-attention for Visual Recognition Kunchang Li Yali Wang Junhao Zhang Peng Gao Guanglu Song Yu Liu Hongsheng Li Yu Qiao ViT 162 360 0 24 Jan 2022
QuadTree Attention for Vision Transformers Shitao Tang Jiahui Zhang Siyu Zhu Ping Tan ViT 169 156 0 08 Jan 2022
UFO-ViT: High Performance Linear Vision Transformer without Softmax Jeonggeun Song ViT 114 20 0 29 Sep 2021
Distract Your Attention: Multi-head Cross Attention Network for Facial Expression Recognition Zhengyao Wen Wen-Long Lin Tao Wang Ge Xu CVBM 107 208 0 15 Sep 2021
MLP-Mixer: An all-MLP Architecture for Vision Ilya O. Tolstikhin N. Houlsby Alexander Kolesnikov Lucas Beyer Xiaohua Zhai ... Andreas Steiner Daniel Keysers Jakob Uszkoreit Mario Lucic Alexey Dosovitskiy 274 2,603 0 04 May 2021
Visformer: The Vision-friendly Transformer Zhengsu Chen Lingxi Xie Jianwei Niu Xuefeng Liu Longhui Wei Qi Tian ViT 120 209 0 26 Apr 2021
Transformer in Transformer Kai Han An Xiao Enhua Wu Jianyuan Guo Chunjing Xu Yunhe Wang ViT 289 1,524 0 27 Feb 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 301 3,623 0 24 Feb 2021
LambdaNetworks: Modeling Long-Range Interactions Without Attention Irwan Bello 278 179 0 17 Feb 2021
High-Performance Large-Scale Image Recognition Without Normalization Andrew Brock Soham De Samuel L. Smith Karen Simonyan VLM 223 512 0 11 Feb 2021
Bottleneck Transformers for Visual Recognition A. Srinivas Nayeon Lee Niki Parmar Jonathon Shlens Pieter Abbeel Ashish Vaswani SLR 290 979 0 27 Jan 2021
Deep High-Resolution Representation Learning for Visual Recognition Jingdong Wang Ke Sun Tianheng Cheng Borui Jiang Chaorui Deng ... Yadong Mu Mingkui Tan Xinggang Wang Wenyu Liu Bin Xiao 195 3,531 0 20 Aug 2019
Aggregated Residual Transformations for Deep Neural Networks Saining Xie Ross B. Girshick Piotr Dollár Z. Tu Kaiming He 297 10,220 0 16 Nov 2016