Contextual Transformer Networks for Visual Recognition

26 July 2021

Yehao Li

Ting Yao

Yingwei Pan

Tao Mei

ViT

ArXiv (abs)PDF HTML Github (532★)

Papers citing "Contextual Transformer Networks for Visual Recognition"

50 / 59 papers shown

Title
Quantum Complex-Valued Self-Attention Model Fu Chen Qinglin Zhao Li Feng Longfei Tang Yangbin Lin Haitao Huang MQ 127 0 0 24 Mar 2025
Breaking the Low-Rank Dilemma of Linear Attention Qihang Fan Huaibo Huang Ran He 101 2 0 12 Nov 2024
Approximate Nullspace Augmented Finetuning for Robust Vision Transformers Haoyang Liu Aditya Singh Yijiang Li Haohan Wang AAML ViT 106 1 0 15 Mar 2024
Progressive Feedback-Enhanced Transformer for Image Forgery Localization Haochen Zhu Gang Cao Xianglin Huang ViT 98 7 0 15 Nov 2023
XCiT: Cross-Covariance Image Transformers Alaaeldin El-Nouby Hugo Touvron Mathilde Caron Piotr Bojanowski Matthijs Douze ... Ivan Laptev Natalia Neverova Gabriel Synnaeve Jakob Verbeek Hervé Jégou ViT 148 513 0 17 Jun 2021
Twins: Revisiting the Design of Spatial Attention in Vision Transformers Xiangxiang Chu Zhi Tian Yuqing Wang Bo Zhang Haibing Ren Xiaolin K. Wei Huaxia Xia Chunhua Shen ViT 84 1,026 0 28 Apr 2021
Co-Scale Conv-Attentional Image Transformers Weijian Xu Yifan Xu Tyler A. Chang Zhuowen Tu ViT 54 376 0 13 Apr 2021
Going deeper with Image Transformers Hugo Touvron Matthieu Cord Alexandre Sablayrolles Gabriel Synnaeve Hervé Jégou ViT 160 1,021 0 31 Mar 2021
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows Ze Liu Yutong Lin Yue Cao Han Hu Yixuan Wei Zheng Zhang Stephen Lin B. Guo ViT 465 21,566 0 25 Mar 2021
ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases Stéphane dÁscoli Hugo Touvron Matthew L. Leavitt Ari S. Morcos Giulio Biroli Levent Sagun ViT 136 833 0 19 Mar 2021
Revisiting ResNets: Improved Training and Scaling Strategies Irwan Bello W. Fedus Xianzhi Du E. D. Cubuk A. Srinivas Nayeon Lee Jonathon Shlens Barret Zoph 87 301 0 13 Mar 2021
Transformer in Transformer Kai Han An Xiao Enhua Wu Jianyuan Guo Chunjing Xu Yunhe Wang ViT 391 1,572 0 27 Feb 2021
LambdaNetworks: Modeling Long-Range Interactions Without Attention Irwan Bello 338 181 0 17 Feb 2021
Bottleneck Transformers for Visual Recognition A. Srinivas Nayeon Lee Niki Parmar Jonathon Shlens Pieter Abbeel Ashish Vaswani SLR 360 994 0 27 Jan 2021
Scheduled Sampling in Vision-Language Pretraining with Decoupled Encoder-Decoder Network Yehao Li Yingwei Pan Ting Yao Jingwen Chen Tao Mei VLM 74 52 0 27 Jan 2021
Training data-efficient image transformers & distillation through attention Hugo Touvron Matthieu Cord Matthijs Douze Francisco Massa Alexandre Sablayrolles Hervé Jégou ViT 389 6,802 0 23 Dec 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 673 41,430 0 22 Oct 2020
Tied Block Convolution: Leaner and Better CNNs with Shared Thinner Filters Xudong Wang Stella X. Yu 45 38 0 25 Sep 2020
Disentangled Non-Local Neural Networks Minghao Yin Zhuliang Yao Yue Cao Xiu Li Zheng Zhang Stephen Lin Han Hu 122 328 0 11 Jun 2020
End-to-End Object Detection with Transformers Nicolas Carion Francisco Massa Gabriel Synnaeve Nicolas Usunier Alexander Kirillov Sergey Zagoruyko ViT 3DV PINN 434 13,108 0 26 May 2020
Exploring Self-attention for Image Recognition Hengshuang Zhao Jiaya Jia V. Koltun SSL 95 786 0 28 Apr 2020
ResNeSt: Split-Attention Networks Hang Zhang Chongruo Wu Zhongyue Zhang Yi Zhu Yanghua Peng ... Tong He Jonas W. Mueller R. Manmatha Mu Li Alex Smola 114 1,480 0 19 Apr 2020
X-Linear Attention Networks for Image Captioning Yingwei Pan Ting Yao Yehao Li Tao Mei 116 513 0 31 Mar 2020
Rethinking Depthwise Separable Convolutions: How Intra-Kernel Correlations Lead to Improved MobileNets D. Haase Manuel Amthor 52 133 0 30 Mar 2020
Dynamic Region-Aware Convolution Jin Chen Xijun Wang Zichao Guo Xinming Zhang Jian Sun 87 118 0 27 Mar 2020
RandAugment: Practical automated data augmentation with a reduced search space E. D. Cubuk Barret Zoph Jonathon Shlens Quoc V. Le MQ 258 3,502 0 30 Sep 2019
Deep High-Resolution Representation Learning for Visual Recognition Jingdong Wang Ke Sun Tianheng Cheng Borui Jiang Chaorui Deng ... Yadong Mu Mingkui Tan Xinggang Wang Wenyu Liu Bin Xiao 393 3,627 0 20 Aug 2019
Stand-Alone Self-Attention in Vision Models Prajit Ramachandran Niki Parmar Ashish Vaswani Irwan Bello Anselm Levskaya Jonathon Shlens VLM SLR ViT 104 1,216 0 13 Jun 2019
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks Mingxing Tan Quoc V. Le 3DV MedIm 161 18,179 0 28 May 2019
GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond Yue Cao Jiarui Xu Stephen Lin Fangyun Wei Han Hu ISeg 86 1,573 0 25 Apr 2019
Local Relation Networks for Image Recognition Han Hu Zheng Zhang Zhenda Xie Stephen Lin FAtt 96 501 0 25 Apr 2019
Attention Augmented Convolutional Networks Irwan Bello Barret Zoph Ashish Vaswani Jonathon Shlens Quoc V. Le 143 1,015 0 22 Apr 2019
Res2Net: A New Multi-scale Backbone Architecture Shanghua Gao Ming-Ming Cheng Kai Zhao Xinyu Zhang Ming-Hsuan Yang Philip Torr 109 2,396 0 02 Apr 2019
Selective Kernel Networks Xiang Li Wenhai Wang Xiaolin Hu Jian Yang 94 2,037 0 15 Mar 2019
Bag of Tricks for Image Classification with Convolutional Neural Networks Tong He Zhi-Li Zhang Hang Zhang Zhongyue Zhang Junyuan Xie Mu Li 293 1,421 0 04 Dec 2018
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 95,175 0 11 Oct 2018
Unified Perceptual Parsing for Scene Understanding Tete Xiao Yingcheng Liu Bolei Zhou Yuning Jiang Jian Sun OCL VOS 195 1,895 0 26 Jul 2018
Context Encoding for Semantic Segmentation Hang Zhang Kristin J. Dana Jianping Shi Zhongyue Zhang Xiaogang Wang A. Tyagi Amit Agrawal SSeg 99 1,251 0 23 Mar 2018
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation Liang-Chieh Chen Yukun Zhu George Papandreou Florian Schroff Hartwig Adam SSeg 480 13,178 0 07 Feb 2018
Cascade R-CNN: Delving into High Quality Object Detection Zhaowei Cai Nuno Vasconcelos ObjD 147 4,941 0 03 Dec 2017
Non-local Neural Networks Xinyu Wang Ross B. Girshick Abhinav Gupta Kaiming He OffRL 300 8,917 0 21 Nov 2017
mixup: Beyond Empirical Risk Minimization Hongyi Zhang Moustapha Cissé Yann N. Dauphin David Lopez-Paz NoLa 289 9,803 0 25 Oct 2017
Squeeze-and-Excitation Networks Jie Hu Li Shen Samuel Albanie Gang Sun Enhua Wu 427 26,557 0 05 Sep 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 786 132,363 0 12 Jun 2017
Mask R-CNN Kaiming He Georgia Gkioxari Piotr Dollár Ross B. Girshick ObjD 366 27,244 0 20 Mar 2017
Deformable Convolutional Networks Jifeng Dai Haozhi Qi Yuwen Xiong Yi Li Guodong Zhang Han Hu Yichen Wei 206 5,339 0 17 Mar 2017
Feature Pyramid Networks for Object Detection Nayeon Lee Piotr Dollár Ross B. Girshick Kaiming He Bharath Hariharan Serge J. Belongie ObjD 485 22,143 0 09 Dec 2016
Pyramid Scene Parsing Network Hengshuang Zhao Jianping Shi Xiaojuan Qi Xiaogang Wang Jiaya Jia VOS SSeg 665 12,033 0 04 Dec 2016
Aggregated Residual Transformations for Deep Neural Networks Saining Xie Ross B. Girshick Piotr Dollár Zhuowen Tu Kaiming He 522 10,347 0 16 Nov 2016
Xception: Deep Learning with Depthwise Separable Convolutions François Chollet MDE BDL PINN 1.4K 14,608 0 07 Oct 2016