Armour: Generalizable Compact Self-Attention for Vision Transformers

3 August 2021

Papers citing "Armour: Generalizable Compact Self-Attention for Vision Transformers"

24 / 24 papers shown

Title
LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inference Ben Graham Alaaeldin El-Nouby Hugo Touvron Pierre Stock Armand Joulin Hervé Jégou Matthijs Douze ViT 76 788 0 02 Apr 2021
CvT: Introducing Convolutions to Vision Transformers Haiping Wu Bin Xiao Noel Codella Mengchen Liu Xiyang Dai Lu Yuan Lei Zhang ViT 152 1,910 0 29 Mar 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 527 3,724 0 24 Feb 2021
Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet Li-xin Yuan Yunpeng Chen Tao Wang Weihao Yu Yujun Shi Zihang Jiang Francis E. H. Tay Jiashi Feng Shuicheng Yan ViT 133 1,939 0 28 Jan 2021
Training data-efficient image transformers & distillation through attention Hugo Touvron Matthieu Cord Matthijs Douze Francisco Massa Alexandre Sablayrolles Hervé Jégou ViT 384 6,768 0 23 Dec 2020
SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning Hanrui Wang Zhekai Zhang Song Han 113 390 0 17 Dec 2020
Pre-Trained Image Processing Transformer Hanting Chen Yunhe Wang Tianyu Guo Chang Xu Yiping Deng Zhenhua Liu Siwei Ma Chunjing Xu Chao Xu Wen Gao VLM ViT 138 1,676 0 01 Dec 2020
Rethinking Transformer-based Set Prediction for Object Detection Zhiqing Sun Shengcao Cao Yiming Yang Kris Kitani ViT 120 322 0 21 Nov 2020
Deformable DETR: Deformable Transformers for End-to-End Object Detection Xizhou Zhu Weijie Su Lewei Lu Bin Li Xiaogang Wang Jifeng Dai ViT 218 5,080 0 08 Oct 2020
Rethinking Attention with Performers K. Choromanski Valerii Likhosherstov David Dohan Xingyou Song Andreea Gane ... Afroz Mohiuddin Lukasz Kaiser David Belanger Lucy J. Colwell Adrian Weller 181 1,585 0 30 Sep 2020
Learning Joint Spatial-Temporal Transformations for Video Inpainting Yanhong Zeng Jianlong Fu Hongyang Chao ViT 92 294 0 20 Jul 2020
Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention Angelos Katharopoulos Apoorv Vyas Nikolaos Pappas Franccois Fleuret 201 1,771 0 29 Jun 2020
Linformer: Self-Attention with Linear Complexity Sinong Wang Belinda Z. Li Madian Khabsa Han Fang Hao Ma 216 1,706 0 08 Jun 2020
Learning Texture Transformer Network for Image Super-Resolution Fuzhi Yang Huan Yang Jianlong Fu Hongtao Lu B. Guo SupR ViT 74 724 0 07 Jun 2020
End-to-End Object Detection with Transformers Nicolas Carion Francisco Massa Gabriel Synnaeve Nicolas Usunier Alexander Kirillov Sergey Zagoruyko ViT 3DV PINN 405 13,048 0 26 May 2020
Reformer: The Efficient Transformer Nikita Kitaev Lukasz Kaiser Anselm Levskaya VLM 189 2,320 0 13 Jan 2020
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks Mingxing Tan Quoc V. Le 3DV MedIm 139 18,134 0 28 May 2019
Generating Long Sequences with Sparse Transformers R. Child Scott Gray Alec Radford Ilya Sutskever 125 1,899 0 23 Apr 2019
End-to-End Dense Video Captioning with Masked Transformer Luowei Zhou Yingbo Zhou Jason J. Corso R. Socher Caiming Xiong 92 528 0 03 Apr 2018
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices Xiangyu Zhang Xinyu Zhou Mengxiao Lin Jian Sun AI4TS 141 6,872 0 04 Jul 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 701 131,652 0 12 Jun 2017
Layer Normalization Jimmy Lei Ba J. Kiros Geoffrey E. Hinton 413 10,494 0 21 Jul 2016
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift Sergey Ioffe Christian Szegedy OOD 463 43,305 0 11 Feb 2015
Neural Machine Translation by Jointly Learning to Align and Translate Dzmitry Bahdanau Kyunghyun Cho Yoshua Bengio AIMat 558 27,311 0 01 Sep 2014