Scaling Local Self-Attention for Parameter Efficient Visual Backbones

23 March 2021

Papers citing "Scaling Local Self-Attention for Parameter Efficient Visual Backbones"

46 / 96 papers shown

Title
Focal Modulation Networks Jianwei Yang Chunyuan Li Xiyang Dai Lu Yuan Jianfeng Gao 3DPC 33 263 0 22 Mar 2022
ScalableViT: Rethinking the Context-oriented Generalization of Vision Transformer Rui Yang Hailong Ma Jie Wu Yansong Tang Xuefeng Xiao Min Zheng Xiu Li ViT 19 53 0 21 Mar 2022
V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformer Runsheng Xu Hao Xiang Zhengzhong Tu Xin Xia Ming-Hsuan Yang Jiaqi Ma ViT 117 362 0 20 Mar 2022
EDTER: Edge Detection with Transformer Mengyang Pu Yaping Huang Yuming Liu Q. Guan Haibin Ling ViT 13 98 0 16 Mar 2022
Smoothing Matters: Momentum Transformer for Domain Adaptive Semantic Segmentation Runfa Chen Yu Rong Shangmin Guo Jiaqi Han Gang Hua Tingyang Xu Wenbing Huang ViT 15 20 0 15 Mar 2022
Scaling Up Your Kernels to 31x31: Revisiting Large Kernel Design in CNNs Xiaohan Ding Xinming Zhang Yi Zhou Jungong Han Guiguang Ding Jian Sun VLM 49 528 0 13 Mar 2022
Boosting Crowd Counting via Multifaceted Attention Hui Lin Zhiheng Ma Rongrong Ji Yaowei Wang Xiaopeng Hong 23 145 0 05 Mar 2022
Aggregating Global Features into Local Vision Transformer Krushi Patel A. Bur Fengju Li Guanghui Wang ViT 33 34 0 30 Jan 2022
ELSA: Enhanced Local Self-Attention for Vision Transformer Jingkai Zhou Pichao Wang Fan Wang Qiong Liu Hao Li Rong Jin ViT 37 37 0 23 Dec 2021
StyleSwin: Transformer-based GAN for High-resolution Image Generation Bo Zhang Shuyang Gu Bo Zhang Jianmin Bao Dong Chen Fang Wen Yong Wang B. Guo ViT 38 223 0 20 Dec 2021
On Efficient Transformer-Based Image Pre-training for Low-Level Vision Wenbo Li Xin Lu Shengju Qian Jiangbo Lu Xinming Zhang Jiaya Jia ViT 32 85 0 19 Dec 2021
Fast Point Transformer Chunghyun Park Yoonwoo Jeong Minsu Cho Jaesik Park 3DPC ViT 38 168 0 09 Dec 2021
PointMixer: MLP-Mixer for Point Cloud Understanding Jaesung Choe Chunghyun Park François Rameau Jaesik Park In So Kweon 3DPC 45 98 0 22 Nov 2021
A Survey of Visual Transformers Yang Liu Yao Zhang Yixin Wang Feng Hou Jin Yuan Jiang Tian Yang Zhang Zhongchao Shi Jianping Fan Zhiqiang He 3DGS ViT 77 330 0 11 Nov 2021
Palette: Image-to-Image Diffusion Models Chitwan Saharia William Chan Huiwen Chang Chris A. Lee Jonathan Ho Tim Salimans David J. Fleet Mohammad Norouzi DiffM VLM 342 1,593 0 10 Nov 2021
Relational Self-Attention: What's Missing in Attention for Video Understanding Manjin Kim Heeseung Kwon Chunyu Wang Suha Kwak Minsu Cho ViT 27 28 0 02 Nov 2021
PatchFormer: An Efficient Point Transformer with Patch Attention Zhang Cheng Haocheng Wan Xinyi Shen Zizhao Wu 3DPC 16 65 0 30 Oct 2021
The Efficiency Misnomer Daoyuan Chen Liuyi Yao Dawei Gao Ashish Vaswani Yaliang Li 34 99 0 25 Oct 2021
UniNet: Unified Architecture Search with Convolution, Transformer, and MLP Jihao Liu Hongsheng Li Guanglu Song Xin Huang Yu Liu ViT 37 35 0 08 Oct 2021
ResNet strikes back: An improved training procedure in timm Ross Wightman Hugo Touvron Hervé Jégou AI4TS 212 487 0 01 Oct 2021
SwinIR: Image Restoration Using Swin Transformer Christos Sakaridis Jie Cao Guolei Sun Kaicheng Zhang Luc Van Gool Radu Timofte ViT 69 2,818 0 23 Aug 2021
Causal Attention for Unbiased Visual Recognition Tan Wang Chan Zhou Qianru Sun Hanwang Zhang OOD CML 32 108 0 19 Aug 2021
Mobile-Former: Bridging MobileNet and Transformer Yinpeng Chen Xiyang Dai Dongdong Chen Mengchen Liu Xiaoyi Dong Lu Yuan Zicheng Liu ViT 183 476 0 12 Aug 2021
Visual Parser: Representing Part-whole Hierarchies with Transformers Shuyang Sun Xiaoyu Yue S. Bai Philip Torr 50 27 0 13 Jul 2021
Focal Self-attention for Local-Global Interactions in Vision Transformers Jianwei Yang Chunyuan Li Pengchuan Zhang Xiyang Dai Bin Xiao Lu Yuan Jianfeng Gao ViT 42 428 0 01 Jul 2021
VOLO: Vision Outlooker for Visual Recognition Li-xin Yuan Qibin Hou Zihang Jiang Jiashi Feng Shuicheng Yan ViT 52 314 0 24 Jun 2021
Vision Permutator: A Permutable MLP-Like Architecture for Visual Recognition Qibin Hou Zihang Jiang Li-xin Yuan Mingg-Ming Cheng Shuicheng Yan Jiashi Feng ViT MLLM 24 205 0 23 Jun 2021
Probabilistic Attention for Interactive Segmentation Prasad Gabbur Manjot Bilkhu J. Movellan 34 13 0 23 Jun 2021
Efficient Self-supervised Vision Transformers for Representation Learning Chunyuan Li Jianwei Yang Pengchuan Zhang Mei Gao Bin Xiao Xiyang Dai Lu Yuan Jianfeng Gao ViT 37 209 0 17 Jun 2021
Improved Transformer for High-Resolution GANs Long Zhao Zizhao Zhang Ting Chen Dimitris N. Metaxas Han Zhang ViT 34 95 0 14 Jun 2021
CoAtNet: Marrying Convolution and Attention for All Data Sizes Zihang Dai Hanxiao Liu Quoc V. Le Mingxing Tan ViT 49 1,167 0 09 Jun 2021
Scaling Vision Transformers Xiaohua Zhai Alexander Kolesnikov N. Houlsby Lucas Beyer ViT 73 1,060 0 08 Jun 2021
On the Connection between Local Attention and Dynamic Depth-wise Convolution Qi Han Zejia Fan Qi Dai Lei-huan Sun Ming-Ming Cheng Jiaying Liu Jingdong Wang ViT 24 105 0 08 Jun 2021
Analogous to Evolutionary Algorithm: Designing a Unified Sequence Model Jiangning Zhang Chao Xu Jian Li Wenzhou Chen Yabiao Wang Ying Tai Shuo Chen Chengjie Wang Feiyue Huang Yong Liu 35 22 0 31 May 2021
Intriguing Properties of Vision Transformers Muzammal Naseer Kanchana Ranasinghe Salman Khan Munawar Hayat Fahad Shahbaz Khan Ming-Hsuan Yang ViT 265 621 0 21 May 2021
Pay Attention to MLPs Hanxiao Liu Zihang Dai David R. So Quoc V. Le AI4CE 57 651 0 17 May 2021
MLP-Mixer: An all-MLP Architecture for Vision Ilya O. Tolstikhin N. Houlsby Alexander Kolesnikov Lucas Beyer Xiaohua Zhai ... Andreas Steiner Daniel Keysers Jakob Uszkoreit Mario Lucic Alexey Dosovitskiy 277 2,606 0 04 May 2021
Visformer: The Vision-friendly Transformer Zhengsu Chen Lingxi Xie Jianwei Niu Xuefeng Liu Longhui Wei Qi Tian ViT 120 209 0 26 Apr 2021
Multi-Scale Vision Longformer: A New Vision Transformer for High-Resolution Image Encoding Pengchuan Zhang Xiyang Dai Jianwei Yang Bin Xiao Lu Yuan Lei Zhang Jianfeng Gao ViT 29 329 0 29 Mar 2021
LambdaNetworks: Modeling Long-Range Interactions Without Attention Irwan Bello 281 179 0 17 Feb 2021
Bottleneck Transformers for Visual Recognition A. Srinivas Nayeon Lee Niki Parmar Jonathon Shlens Pieter Abbeel Ashish Vaswani SLR 290 980 0 27 Jan 2021
Transformers in Vision: A Survey Salman Khan Muzammal Naseer Munawar Hayat Syed Waqas Zamir Fahad Shahbaz Khan M. Shah ViT 227 2,431 0 04 Jan 2021
Efficient Content-Based Sparse Attention with Routing Transformers Aurko Roy M. Saffar Ashish Vaswani David Grangier MoE 252 580 0 12 Mar 2020
Bag of Tricks for Image Classification with Convolutional Neural Networks Tong He Zhi-Li Zhang Hang Zhang Zhongyue Zhang Junyuan Xie Mu Li 221 1,399 0 04 Dec 2018
Aggregated Residual Transformations for Deep Neural Networks Saining Xie Ross B. Girshick Piotr Dollár Z. Tu Kaiming He 297 10,225 0 16 Nov 2016
A Decomposable Attention Model for Natural Language Inference Ankur P. Parikh Oscar Täckström Dipanjan Das Jakob Uszkoreit 213 1,367 0 06 Jun 2016