MaiT: Leverage Attention Masks for More Efficient Image Transformers

6 July 2022

Papers citing "MaiT: Leverage Attention Masks for More Efficient Image Transformers"

14 / 14 papers shown

Title
MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer Sachin Mehta Mohammad Rastegari ViT 276 1,257 0 05 Oct 2021
Efficient Self-supervised Vision Transformers for Representation Learning Chunyuan Li Jianwei Yang Pengchuan Zhang Mei Gao Bin Xiao Xiyang Dai Lu Yuan Jianfeng Gao ViT 92 213 0 17 Jun 2021
Going deeper with Image Transformers Hugo Touvron Matthieu Cord Alexandre Sablayrolles Gabriel Synnaeve Hervé Jégou ViT 133 1,006 0 31 Mar 2021
CvT: Introducing Convolutions to Vision Transformers Haiping Wu Bin Xiao Noel Codella Mengchen Liu Xiyang Dai Lu Yuan Lei Zhang ViT 134 1,901 0 29 Mar 2021
CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification Chun-Fu Chen Quanfu Fan Yikang Shen ViT 68 1,469 0 27 Mar 2021
BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search Changlin Li Tao Tang Guangrun Wang Jiefeng Peng Bing Wang Xiaodan Liang Xiaojun Chang ViT 97 107 0 23 Mar 2021
Incorporating Convolution Designs into Visual Transformers Kun Yuan Shaopeng Guo Ziwei Liu Aojun Zhou F. Yu Wei Wu ViT 106 476 0 22 Mar 2021
Perceiver: General Perception with Iterative Attention Andrew Jaegle Felix Gimeno Andrew Brock Andrew Zisserman Oriol Vinyals João Carreira VLM ViT MDE 168 1,014 0 04 Mar 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 495 3,709 0 24 Feb 2021
Bottleneck Transformers for Visual Recognition A. Srinivas Nayeon Lee Niki Parmar Jonathon Shlens Pieter Abbeel Ashish Vaswani SLR 344 989 0 27 Jan 2021
Visual Transformers: Token-based Image Representation and Processing for Computer Vision Bichen Wu Chenfeng Xu Xiaoliang Dai Alvin Wan Peizhao Zhang Zhicheng Yan Masayoshi Tomizuka Joseph E. Gonzalez Kurt Keutzer Peter Vajda ViT 94 558 0 05 Jun 2020
Longformer: The Long-Document Transformer Iz Beltagy Matthew E. Peters Arman Cohan RALM VLM 131 4,048 0 10 Apr 2020
Generating Long Sequences with Sparse Transformers R. Child Scott Gray Alec Radford Ilya Sutskever 95 1,894 0 23 Apr 2019
Revisiting Unreasonable Effectiveness of Data in Deep Learning Era Chen Sun Abhinav Shrivastava Saurabh Singh Abhinav Gupta VLM 170 2,393 0 10 Jul 2017