Rethinking Token-Mixing MLP for MLP-based Vision Backbone

28 June 2021

Papers citing "Rethinking Token-Mixing MLP for MLP-based Vision Backbone"

40 / 40 papers shown

Title
MVT: Multi-view Vision Transformer for 3D Object Recognition Shuo Chen Tan Yu Ping Li ViT 46 44 0 25 Oct 2021
S $^2$ -MLPv2: Improved Spatial-Shift MLP Architecture for Vision Tan Yu Xu Li Yunfeng Cai Mingming Sun Ping Li 51 51 0 02 Aug 2021
S $^2$ -MLP: Spatial-Shift MLP Architecture for Vision Tan Yu Xu Li Yunfeng Cai Mingming Sun Ping Li 53 187 0 14 Jun 2021
Chasing Sparsity in Vision Transformers: An End-to-End Exploration Tianlong Chen Yu Cheng Zhe Gan Lu Yuan Lei Zhang Zhangyang Wang ViT 33 216 0 08 Jun 2021
Shuffle Transformer: Rethinking Spatial Shuffle for Vision Transformer Zilong Huang Youcheng Ben Guozhong Luo Pei Cheng Gang Yu Bin-Bin Fu ViT 32 182 0 07 Jun 2021
Container: Context Aggregation Network Peng Gao Jiasen Lu Hongsheng Li Roozbeh Mottaghi Aniruddha Kembhavi ViT 44 71 0 02 Jun 2021
Pay Attention to MLPs Hanxiao Liu Zihang Dai David R. So Quoc V. Le AI4CE 79 657 0 17 May 2021
ResMLP: Feedforward networks for image classification with data-efficient training Hugo Touvron Piotr Bojanowski Mathilde Caron Matthieu Cord Alaaeldin El-Nouby ... Gautier Izacard Armand Joulin Gabriel Synnaeve Jakob Verbeek Hervé Jégou VLM 52 657 0 07 May 2021
Do You Even Need Attention? A Stack of Feed-Forward Layers Does Surprisingly Well on ImageNet Luke Melas-Kyriazi ViT 18 102 0 06 May 2021
Beyond Self-attention: External Attention using Two Linear Layers for Visual Tasks Meng-Hao Guo Zheng-Ning Liu Tai-Jiang Mu Shimin Hu 48 480 0 05 May 2021
MLP-Mixer: An all-MLP Architecture for Vision Ilya O. Tolstikhin N. Houlsby Alexander Kolesnikov Lucas Beyer Xiaohua Zhai ... Andreas Steiner Daniel Keysers Jakob Uszkoreit Mario Lucic Alexey Dosovitskiy 374 2,638 0 04 May 2021
Twins: Revisiting the Design of Spatial Attention in Vision Transformers Xiangxiang Chu Zhi Tian Yuqing Wang Bo Zhang Haibing Ren Xiaolin K. Wei Huaxia Xia Chunhua Shen ViT 37 1,006 0 28 Apr 2021
Multiscale Vision Transformers Haoqi Fan Bo Xiong K. Mangalam Yanghao Li Zhicheng Yan Jitendra Malik Christoph Feichtenhofer ViT 86 1,239 0 22 Apr 2021
Going deeper with Image Transformers Hugo Touvron Matthieu Cord Alexandre Sablayrolles Gabriel Synnaeve Hervé Jégou ViT 108 998 0 31 Mar 2021
Rethinking Spatial Dimensions of Vision Transformers Byeongho Heo Sangdoo Yun Dongyoon Han Sanghyuk Chun Junsuk Choe Seong Joon Oh ViT 472 573 0 30 Mar 2021
CvT: Introducing Convolutions to Vision Transformers Haiping Wu Bin Xiao Noel Codella Mengchen Liu Xiyang Dai Lu Yuan Lei Zhang ViT 105 1,891 0 29 Mar 2021
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows Ze Liu Yutong Lin Yue Cao Han Hu Yixuan Wei Zheng Zhang Stephen Lin B. Guo ViT 213 21,051 0 25 Mar 2021
Transformer in Transformer Kai Han An Xiao Enhua Wu Jianyuan Guo Chunjing Xu Yunhe Wang ViT 357 1,544 0 27 Feb 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 417 3,660 0 24 Feb 2021
Conditional Positional Encodings for Vision Transformers Xiangxiang Chu Zhi Tian Bo Zhang Xinlong Wang Chunhua Shen ViT 47 607 0 22 Feb 2021
Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet Li-xin Yuan Yunpeng Chen Tao Wang Weihao Yu Yujun Shi Zihang Jiang Francis E. H. Tay Jiashi Feng Shuicheng Yan ViT 80 1,918 0 28 Jan 2021
Training data-efficient image transformers & distillation through attention Hugo Touvron Matthieu Cord Matthijs Douze Francisco Massa Alexandre Sablayrolles Hervé Jégou ViT 238 6,657 0 23 Dec 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 180 40,217 0 22 Oct 2020
Designing Network Design Spaces Ilija Radosavovic Raj Prateek Kosaraju Ross B. Girshick Kaiming He Piotr Dollár GNN 71 1,672 0 30 Mar 2020
4-Connected Shift Residual Networks Andrew Brown Pascal Mettes Marcel Worring 3DPC 38 8 0 22 Oct 2019
RandAugment: Practical automated data augmentation with a reduced search space E. D. Cubuk Barret Zoph Jonathon Shlens Quoc V. Le MQ 169 3,458 0 30 Sep 2019
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks Mingxing Tan Quoc V. Le 3DV MedIm 44 17,950 0 28 May 2019
CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features Sangdoo Yun Dongyoon Han Seong Joon Oh Sanghyuk Chun Junsuk Choe Y. Yoo OOD 557 4,735 0 13 May 2019
Shift: A Zero FLOP, Zero Parameter Alternative to Spatial Convolutions Bichen Wu Alvin Wan Xiangyu Yue Peter H. Jin Sicheng Zhao Noah Golmant A. Gholaminejad Joseph E. Gonzalez Kurt Keutzer 3DPC 49 363 0 22 Nov 2017
mixup: Beyond Empirical Risk Minimization Hongyi Zhang Moustapha Cissé Yann N. Dauphin David Lopez-Paz NoLa 217 9,687 0 25 Oct 2017
Revisiting Unreasonable Effectiveness of Data in Deep Learning Era Chen Sun Abhinav Shrivastava Saurabh Singh Abhinav Gupta VLM 87 2,378 0 10 Jul 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 319 129,831 0 12 Jun 2017
Depthwise Separable Convolutions for Neural Machine Translation Lukasz Kaiser Aidan Gomez François Chollet 52 279 0 09 Jun 2017
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications Andrew G. Howard Menglong Zhu Bo Chen Dmitry Kalenichenko Weijun Wang Tobias Weyand M. Andreetto Hartwig Adam 3DH 1.0K 20,692 0 17 Apr 2017
Xception: Deep Learning with Depthwise Separable Convolutions François Chollet MDE BDL PINN 745 14,454 0 07 Oct 2016
Layer Normalization Jimmy Lei Ba J. Kiros Geoffrey E. Hinton 199 10,412 0 21 Jul 2016
Gaussian Error Linear Units (GELUs) Dan Hendrycks Kevin Gimpel 142 4,934 0 27 Jun 2016
FractalNet: Ultra-Deep Neural Networks without Residuals Gustav Larsson Michael Maire Gregory Shakhnarovich 90 934 0 24 May 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 1.1K 192,638 0 10 Dec 2015
Rethinking the Inception Architecture for Computer Vision Christian Szegedy Vincent Vanhoucke Sergey Ioffe Jonathon Shlens Z. Wojna 3DV BDL 390 27,231 0 02 Dec 2015