ScalableViT: Rethinking the Context-oriented Generalization of Vision Transformer

21 March 2022

Min Zheng

Papers citing "ScalableViT: Rethinking the Context-oriented Generalization of Vision Transformer"

50 / 53 papers shown

Title
Breaking the Low-Rank Dilemma of Linear Attention Qihang Fan Huaibo Huang Ran He 89 1 0 12 Nov 2024
FViT: A Focal Vision Transformer with Gabor Filter Yulong Shi Mingwei Sun Yongshuai Wang Rui Wang 104 4 0 17 Feb 2024
TransXNet: Learning Both Global and Local Dynamics with a Dual Dynamic Token Mixer for Visual Recognition Meng Lou Hong-Yu Zhou Sibei Yang Yizhou Yu Chuan Wu Yizhou Yu ViT 78 40 0 30 Oct 2023
Lightweight Vision Transformer with Bidirectional Interaction Qihang Fan Huaibo Huang Xiaoqiang Zhou Ran He ViT 105 29 0 01 Jun 2023
TRT-ViT: TensorRT-oriented Vision Transformer Xin Xia Jiashi Li Jie Wu Xing Wang Xuefeng Xiao Min Zheng Rui Wang ViT 42 28 0 19 May 2022
HRFormer: High-Resolution Transformer for Dense Prediction Yuhui Yuan Rao Fu Lang Huang Weihong Lin Chao Zhang Xilin Chen Jingdong Wang ViT 76 233 0 18 Oct 2021
CrossFormer: A Versatile Vision Transformer Hinging on Cross-scale Attention Wenxiao Wang Lulian Yao Long Chen Binbin Lin Deng Cai Xiaofei He Wei Liu 169 270 0 31 Jul 2021
CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windows Xiaoyi Dong Jianmin Bao Dongdong Chen Weiming Zhang Nenghai Yu Lu Yuan Dong Chen B. Guo ViT 139 981 0 01 Jul 2021
Shuffle Transformer: Rethinking Spatial Shuffle for Vision Transformer Zilong Huang Youcheng Ben Guozhong Luo Pei Cheng Gang Yu Bin-Bin Fu ViT 66 182 0 07 Jun 2021
DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification Yongming Rao Wenliang Zhao Benlin Liu Jiwen Lu Jie Zhou Cho-Jui Hsieh ViT 76 697 0 03 Jun 2021
Twins: Revisiting the Design of Spatial Attention in Vision Transformers Xiangxiang Chu Zhi Tian Yuqing Wang Bo Zhang Haibing Ren Xiaolin K. Wei Huaxia Xia Chunhua Shen ViT 79 1,017 0 28 Apr 2021
Co-Scale Conv-Attentional Image Transformers Weijian Xu Yifan Xu Tyler A. Chang Zhuowen Tu ViT 52 375 0 13 Apr 2021
CvT: Introducing Convolutions to Vision Transformers Haiping Wu Bin Xiao Noel Codella Mengchen Liu Xiyang Dai Lu Yuan Lei Zhang ViT 145 1,907 0 29 Mar 2021
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows Ze Liu Yutong Lin Yue Cao Han Hu Yixuan Wei Zheng Zhang Stephen Lin B. Guo ViT 434 21,392 0 25 Mar 2021
Scaling Local Self-Attention for Parameter Efficient Visual Backbones Ashish Vaswani Prajit Ramachandran A. Srinivas Niki Parmar Blake A. Hechtman Jonathon Shlens 85 400 0 23 Mar 2021
Incorporating Convolution Designs into Visual Transformers Kun Yuan Shaopeng Guo Ziwei Liu Aojun Zhou F. Yu Wei Wu ViT 108 477 0 22 Mar 2021
Transformer in Transformer Kai Han An Xiao Enhua Wu Jianyuan Guo Chunjing Xu Yunhe Wang ViT 380 1,561 0 27 Feb 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 516 3,718 0 24 Feb 2021
Conditional Positional Encodings for Vision Transformers Xiangxiang Chu Zhi Tian Bo Zhang Xinlong Wang Chunhua Shen ViT 74 616 0 22 Feb 2021
Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet Li-xin Yuan Yunpeng Chen Tao Wang Weihao Yu Yujun Shi Zihang Jiang Francis E. H. Tay Jiashi Feng Shuicheng Yan ViT 124 1,935 0 28 Jan 2021
Training data-efficient image transformers & distillation through attention Hugo Touvron Matthieu Cord Matthijs Douze Francisco Massa Alexandre Sablayrolles Hervé Jégou ViT 372 6,757 0 23 Dec 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 601 40,961 0 22 Oct 2020
Disentangled Non-Local Neural Networks Minghao Yin Zhuliang Yao Yue Cao Xiu Li Zheng Zhang Stephen Lin Han Hu 99 328 0 11 Jun 2020
End-to-End Object Detection with Transformers Nicolas Carion Francisco Massa Gabriel Synnaeve Nicolas Usunier Alexander Kirillov Sergey Zagoruyko ViT 3DV PINN 372 13,025 0 26 May 2020
Exploring Self-attention for Image Recognition Hengshuang Zhao Jiaya Jia V. Koltun SSL 93 786 0 28 Apr 2020
Designing Network Design Spaces Ilija Radosavovic Raj Prateek Kosaraju Ross B. Girshick Kaiming He Piotr Dollár GNN 100 1,682 0 30 Mar 2020
Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation Huiyu Wang Yukun Zhu Bradley Green Hartwig Adam Alan Yuille Liang-Chieh Chen 3DPC 114 671 0 17 Mar 2020
How Much Position Information Do Convolutional Neural Networks Encode? Md. Amirul Islam Sen Jia Neil D. B. Bruce SSL 244 348 0 22 Jan 2020
Interlaced Sparse Self-Attention for Semantic Segmentation Lang Huang Yuhui Yuan Jianyuan Guo Chao Zhang Xilin Chen Jingdong Wang 66 155 0 29 Jul 2019
MMDetection: Open MMLab Detection Toolbox and Benchmark Kai-xiang Chen Jiaqi Wang Jiangmiao Pang Yuhang Cao Yu Xiong ... Jingdong Wang Jianping Shi Wanli Ouyang Chen Change Loy Dahua Lin VOS 135 2,866 0 17 Jun 2019
Stand-Alone Self-Attention in Vision Models Prajit Ramachandran Niki Parmar Ashish Vaswani Irwan Bello Anselm Levskaya Jonathon Shlens VLM SLR ViT 89 1,213 0 13 Jun 2019
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks Mingxing Tan Quoc V. Le 3DV MedIm 133 18,106 0 28 May 2019
CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features Sangdoo Yun Dongyoon Han Seong Joon Oh Sanghyuk Chun Junsuk Choe Y. Yoo OOD 606 4,777 0 13 May 2019
GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond Yue Cao Jiarui Xu Stephen Lin Fangyun Wei Han Hu ISeg 76 1,570 0 25 Apr 2019
Local Relation Networks for Image Recognition Han Hu Zheng Zhang Zhenda Xie Stephen Lin FAtt 67 501 0 25 Apr 2019
Attention Augmented Convolutional Networks Irwan Bello Barret Zoph Ashish Vaswani Jonathon Shlens Quoc V. Le 132 1,014 0 22 Apr 2019
Panoptic Feature Pyramid Networks Alexander Kirillov Ross B. Girshick Kaiming He Piotr Dollár ISeg SSeg 110 1,285 0 08 Jan 2019
CCNet: Criss-Cross Attention for Semantic Segmentation Zilong Huang Xinggang Wang Yunchao Wei Lichao Huang Humphrey Shi Wenyu Liu Chang Huang VOS 210 2,544 0 28 Nov 2018
Unified Perceptual Parsing for Scene Understanding Tete Xiao Yingcheng Liu Bolei Zhou Yuning Jiang Jian Sun OCL VOS 187 1,883 0 26 Jul 2018
Non-local Neural Networks Xinyu Wang Ross B. Girshick Abhinav Gupta Kaiming He OffRL 283 8,902 0 21 Nov 2017
mixup: Beyond Empirical Risk Minimization Hongyi Zhang Moustapha Cissé Yann N. Dauphin David Lopez-Paz NoLa 273 9,759 0 25 Oct 2017
Random Erasing Data Augmentation Zhun Zhong Liang Zheng Guoliang Kang Shaozi Li Yi Yang 90 3,634 0 16 Aug 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 677 131,414 0 12 Jun 2017
Mask R-CNN Kaiming He Georgia Gkioxari Piotr Dollár Ross B. Girshick ObjD 348 27,174 0 20 Mar 2017
Feature Pyramid Networks for Object Detection Nayeon Lee Piotr Dollár Ross B. Girshick Kaiming He Bharath Hariharan Serge J. Belongie ObjD 453 22,095 0 09 Dec 2016
Aggregated Residual Transformations for Deep Neural Networks Saining Xie Ross B. Girshick Piotr Dollár Zhuowen Tu Kaiming He 506 10,318 0 16 Nov 2016
Semantic Understanding of Scenes through the ADE20K Dataset Bolei Zhou Hang Zhao Xavier Puig Tete Xiao Sanja Fidler Adela Barriuso Antonio Torralba SSeg 396 1,871 0 18 Aug 2016
Deep Networks with Stochastic Depth Gao Huang Yu Sun Zhuang Liu Daniel Sedra Kilian Q. Weinberger 209 2,356 0 30 Mar 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.2K 193,814 0 10 Dec 2015
Rethinking the Inception Architecture for Computer Vision Christian Szegedy Vincent Vanhoucke Sergey Ioffe Jonathon Shlens Z. Wojna 3DV BDL 872 27,350 0 02 Dec 2015