Mixing and Shifting: Exploiting Global and Local Dependencies in Vision MLPs

14 February 2022

Huangjie Zheng

Pengcheng He

Weizhu Chen

Mingyuan Zhou

ArXiv PDF HTML

Papers citing "Mixing and Shifting: Exploiting Global and Local Dependencies in Vision MLPs"

42 / 42 papers shown

Title
A ConvNet for the 2020s Zhuang Liu Hanzi Mao Chaozheng Wu Christoph Feichtenhofer Trevor Darrell Saining Xie ViT 54 5,073 0 10 Jan 2022
An Image Patch is a Wave: Phase-Aware Vision MLP Yehui Tang Kai Han Jianyuan Guo Chang Xu Yanxi Li Chao Xu Yunhe Wang 40 135 0 24 Nov 2021
Sparse MLP for Image Recognition: Is Self-Attention Really Necessary? Chuanxin Tang Yucheng Zhao Guangting Wang Chong Luo Wenxuan Xie Wenjun Zeng MoE ViT 43 98 0 12 Sep 2021
ConvMLP: Hierarchical Convolutional MLPs for Vision Jiachen Li Ali Hassani Steven Walton Humphrey Shi 61 67 0 09 Sep 2021
Hire-MLP: Vision MLP via Hierarchical Rearrangement Jianyuan Guo Yehui Tang Kai Han Xinghao Chen Han Wu Chao Xu Chang Xu Yunhe Wang 67 105 0 30 Aug 2021
Rethinking and Improving Relative Position Encoding for Vision Transformer Kan Wu Houwen Peng Minghao Chen Jianlong Fu Hongyang Chao ViT 66 333 0 29 Jul 2021
CycleMLP: A MLP-like Architecture for Dense Prediction Shoufa Chen Enze Xie Chongjian Ge Runjian Chen Ding Liang Ping Luo 115 231 0 21 Jul 2021
AS-MLP: An Axial Shifted MLP Architecture for Vision Dongze Lian Zehao Yu Xing Sun Shenghua Gao 104 189 0 18 Jul 2021
GLiT: Neural Architecture Search for Global and Local Image Transformer Boyu Chen Peixia Li Chuming Li Baopu Li Lei Bai Chen Lin Ming Sun Junjie Yan Wanli Ouyang ViT 58 85 0 07 Jul 2021
Global Filter Networks for Image Classification Yongming Rao Wenliang Zhao Zheng Zhu Jiwen Lu Jie Zhou ViT 39 458 0 01 Jul 2021
Vision Permutator: A Permutable MLP-Like Architecture for Visual Recognition Qibin Hou Zihang Jiang Li-xin Yuan Mingg-Ming Cheng Shuicheng Yan Jiashi Feng ViT MLLM 90 206 0 23 Jun 2021
S $^2$ -MLP: Spatial-Shift MLP Architecture for Vision Tan Yu Xu Li Yunfeng Cai Mingming Sun Ping Li 53 187 0 14 Jun 2021
Pay Attention to MLPs Hanxiao Liu Zihang Dai David R. So Quoc V. Le AI4CE 77 657 0 17 May 2021
ResMLP: Feedforward networks for image classification with data-efficient training Hugo Touvron Piotr Bojanowski Mathilde Caron Matthieu Cord Alaaeldin El-Nouby ... Gautier Izacard Armand Joulin Gabriel Synnaeve Jakob Verbeek Hervé Jégou VLM 52 657 0 07 May 2021
Do You Even Need Attention? A Stack of Feed-Forward Layers Does Surprisingly Well on ImageNet Luke Melas-Kyriazi ViT 18 102 0 06 May 2021
MLP-Mixer: An all-MLP Architecture for Vision Ilya O. Tolstikhin N. Houlsby Alexander Kolesnikov Lucas Beyer Xiaohua Zhai ... Andreas Steiner Daniel Keysers Jakob Uszkoreit Mario Lucic Alexey Dosovitskiy 371 2,638 0 04 May 2021
CvT: Introducing Convolutions to Vision Transformers Haiping Wu Bin Xiao Noel Codella Mengchen Liu Xiyang Dai Lu Yuan Lei Zhang ViT 105 1,891 0 29 Mar 2021
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows Ze Liu Yutong Lin Yue Cao Han Hu Yixuan Wei Zheng Zhang Stephen Lin B. Guo ViT 208 21,051 0 25 Mar 2021
Transformer in Transformer Kai Han An Xiao Enhua Wu Jianyuan Guo Chunjing Xu Yunhe Wang ViT 354 1,544 0 27 Feb 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 404 3,660 0 24 Feb 2021
Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet Li-xin Yuan Yunpeng Chen Tao Wang Weihao Yu Yujun Shi Zihang Jiang Francis E. H. Tay Jiashi Feng Shuicheng Yan ViT 80 1,918 0 28 Jan 2021
Training data-efficient image transformers & distillation through attention Hugo Touvron Matthieu Cord Matthijs Douze Francisco Massa Alexandre Sablayrolles Hervé Jégou ViT 228 6,657 0 23 Dec 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 161 40,217 0 22 Oct 2020
Designing Network Design Spaces Ilija Radosavovic Raj Prateek Kosaraju Ross B. Girshick Kaiming He Piotr Dollár GNN 68 1,672 0 30 Mar 2020
RandAugment: Practical automated data augmentation with a reduced search space E. D. Cubuk Barret Zoph Jonathon Shlens Quoc V. Le MQ 169 3,458 0 30 Sep 2019
MMDetection: Open MMLab Detection Toolbox and Benchmark Kai-xiang Chen Jiaqi Wang Jiangmiao Pang Yuhang Cao Yu Xiong ... Jingdong Wang Jianping Shi Wanli Ouyang Chen Change Loy Dahua Lin VOS 78 2,845 0 17 Jun 2019
CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features Sangdoo Yun Dongyoon Han Seong Joon Oh Sanghyuk Chun Junsuk Choe Y. Yoo OOD 553 4,735 0 13 May 2019
Unified Perceptual Parsing for Scene Understanding Tete Xiao Yingcheng Liu Bolei Zhou Yuning Jiang Jian Sun OCL VOS 79 1,859 0 26 Jul 2018
Cascade R-CNN: Delving into High Quality Object Detection Zhaowei Cai Nuno Vasconcelos ObjD 94 4,885 0 03 Dec 2017
mixup: Beyond Empirical Risk Minimization Hongyi Zhang Moustapha Cissé Yann N. Dauphin David Lopez-Paz NoLa 212 9,687 0 25 Oct 2017
Random Erasing Data Augmentation Zhun Zhong Liang Zheng Guoliang Kang Shaozi Li Yi Yang 63 3,614 0 16 Aug 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 309 129,831 0 12 Jun 2017
Mask R-CNN Kaiming He Georgia Gkioxari Piotr Dollár Ross B. Girshick ObjD 232 27,018 0 20 Mar 2017
Semantic Understanding of Scenes through the ADE20K Dataset Bolei Zhou Hang Zhao Xavier Puig Tete Xiao Sanja Fidler Adela Barriuso Antonio Torralba SSeg 309 1,850 0 18 Aug 2016
Layer Normalization Jimmy Lei Ba J. Kiros Geoffrey E. Hinton 194 10,412 0 21 Jul 2016
Deep Networks with Stochastic Depth Gao Huang Yu Sun Zhuang Liu Daniel Sedra Kilian Q. Weinberger 119 2,344 0 30 Mar 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 1.1K 192,638 0 10 Dec 2015
Rethinking the Inception Architecture for Computer Vision Christian Szegedy Vincent Vanhoucke Sergey Ioffe Jonathon Shlens Z. Wojna 3DV BDL 373 27,231 0 02 Dec 2015
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift Sergey Ioffe Christian Szegedy OOD 219 43,154 0 11 Feb 2015
Going Deeper with Convolutions Christian Szegedy Wei Liu Yangqing Jia P. Sermanet Scott E. Reed Dragomir Anguelov D. Erhan Vincent Vanhoucke Andrew Rabinovich 235 43,511 0 17 Sep 2014
Very Deep Convolutional Networks for Large-Scale Image Recognition Karen Simonyan Andrew Zisserman FAtt MDE 683 99,991 0 04 Sep 2014
Microsoft COCO: Common Objects in Context Nayeon Lee Michael Maire Serge J. Belongie Lubomir Bourdev Ross B. Girshick James Hays Pietro Perona Deva Ramanan C. L. Zitnick Piotr Dollár ObjD 158 43,290 0 01 May 2014