v1v2 (latest)

LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inference

2 April 2021

Ben Graham

ArXiv (abs)PDF HTML Github (612★)

Papers citing "LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inference"

50 / 68 papers shown

Title
TESSER: Transfer-Enhancing Adversarial Attacks from Vision Transformers via Spectral and Semantic Regularization Amira Guesmi B. Ouni Muhammad Shafique AAML 231 0 0 26 May 2025
AnchorFormer: Differentiable Anchor Attention for Efficient Vision Transformer Jiquan Shan Junxiao Wang Lifeng Zhao Liang Cai Hongyuan Zhang Ioannis Liritzis ViT 228 0 0 22 May 2025
Improving Transferable Targeted Attacks with Feature Tuning Mixup K. Liang Xuelong Dai Yanjie Li Dong Wang Bin Xiao AAML 473 0 0 23 Nov 2024
SAG-ViT: A Scale-Aware, High-Fidelity Patching Approach with Graph Attention for Vision Transformers Shravan Venkatraman Jaskaran Singh Walia J. Raheja ViT 186 0 0 14 Nov 2024
MambaVision: A Hybrid Mamba-Transformer Vision Backbone Ali Hatamizadeh Jan Kautz Mamba 129 70 0 10 Jul 2024
The 3D-PC: a benchmark for visual perspective taking in humans and machines Drew Linsley Peisen Zhou A. Ashok Akash Nagaraj Gaurav Gaonkar Francis E Lewis Zygmunt Pizlo Thomas Serre 115 6 0 06 Jun 2024
OnDev-LCT: On-Device Lightweight Convolutional Transformers towards federated learning Chu Myaet Thwal Minh N. H. Nguyen Ye Lin Tun Seongjin Kim My T. Thai Choong Seon Hong 101 5 0 22 Jan 2024
SCHEME: Scalable Channel Mixer for Vision Transformers Deepak Sridhar Yunsheng Li Nuno Vasconcelos 113 0 0 01 Dec 2023
Associative Transformer Yuwei Sun H. Ochiai Zhirong Wu Stephen Lin Ryota Kanai ViT 94 0 0 22 Sep 2023
ViTCoD: Vision Transformer Acceleration via Dedicated Algorithm and Accelerator Co-Design Haoran You Zhanyi Sun Huihong Shi Zhongzhi Yu Yang Zhao Yongan Zhang Chaojian Li Baopu Li Yingyan Lin ViT 76 83 0 18 Oct 2022
FBNetV5: Neural Architecture Search for Multiple Tasks in One Run Bichen Wu Chaojian Li Hang Zhang Xiaoliang Dai Peizhao Zhang Matthew Yu Jialiang Wang Yingyan Lin Peter Vajda ViT 108 24 0 19 Nov 2021
EfficientNetV2: Smaller Models and Faster Training Mingxing Tan Quoc V. Le EgoV 122 2,714 0 01 Apr 2021
Rethinking Spatial Dimensions of Vision Transformers Byeongho Heo Sangdoo Yun Dongyoon Han Sanghyuk Chun Junsuk Choe Seong Joon Oh ViT 506 582 0 30 Mar 2021
CvT: Introducing Convolutions to Vision Transformers Haiping Wu Bin Xiao Noel Codella Mengchen Liu Xiyang Dai Lu Yuan Lei Zhang ViT 152 1,915 0 29 Mar 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 530 3,734 0 24 Feb 2021
LambdaNetworks: Modeling Long-Range Interactions Without Attention Irwan Bello 335 181 0 17 Feb 2021
High-Performance Large-Scale Image Recognition Without Normalization Andrew Brock Soham De Samuel L. Smith Karen Simonyan VLM 285 522 0 11 Feb 2021
Training Vision Transformers for Image Retrieval Alaaeldin El-Nouby Natalia Neverova Ivan Laptev Hervé Jégou ViT 126 159 0 10 Feb 2021
Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet Li-xin Yuan Yunpeng Chen Tao Wang Weihao Yu Yujun Shi Zihang Jiang Francis E. H. Tay Jiashi Feng Shuicheng Yan ViT 133 1,941 0 28 Jan 2021
Bottleneck Transformers for Visual Recognition A. Srinivas Nayeon Lee Niki Parmar Jonathon Shlens Pieter Abbeel Ashish Vaswani SLR 360 993 0 27 Jan 2021
Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers Sixiao Zheng Jiachen Lu Hengshuang Zhao Xiatian Zhu Zekun Luo ... Yanwei Fu Jianfeng Feng Tao Xiang Philip Torr Li Zhang ViT 194 2,908 0 31 Dec 2020
Training data-efficient image transformers & distillation through attention Hugo Touvron Matthieu Cord Matthijs Douze Francisco Massa Alexandre Sablayrolles Hervé Jégou ViT 389 6,793 0 23 Dec 2020
Toward Transformer-Based Object Detection Josh Beal Eric Kim Eric Tzeng Dong Huk Park Andrew Zhai Dmitry Kislyuk ViT 91 215 0 17 Dec 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 667 41,369 0 22 Oct 2020
Global Self-Attention Networks for Image Recognition Zhuoran Shen Irwan Bello Raviteja Vemulapalli Xuhui Jia Ching-Hui Chen ViT 58 29 0 06 Oct 2020
Are we done with ImageNet? Lucas Beyer Olivier J. Hénaff Alexander Kolesnikov Xiaohua Zhai Aaron van den Oord VLM 126 405 0 12 Jun 2020
Visual Transformers: Token-based Image Representation and Processing for Computer Vision Bichen Wu Chenfeng Xu Xiaoliang Dai Alvin Wan Peizhao Zhang Zhicheng Yan Masayoshi Tomizuka Joseph E. Gonzalez Kurt Keutzer Peter Vajda ViT 101 562 0 05 Jun 2020
Exploring Self-attention for Image Recognition Hengshuang Zhao Jiaya Jia V. Koltun SSL 95 786 0 28 Apr 2020
ResNeSt: Split-Attention Networks Hang Zhang Chongruo Wu Zhongyue Zhang Yi Zhu Yanghua Peng ... Tong He Jonas W. Mueller R. Manmatha Mu Li Alex Smola 108 1,479 0 19 Apr 2020
Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks Xiujun Li Xi Yin Chunyuan Li Pengchuan Zhang Xiaowei Hu ... Houdong Hu Li Dong Furu Wei Yejin Choi Jianfeng Gao VLM 121 1,944 0 13 Apr 2020
Designing Network Design Spaces Ilija Radosavovic Raj Prateek Kosaraju Ross B. Girshick Kaiming He Piotr Dollár GNN 102 1,691 0 30 Mar 2020
Dynamic Convolution: Attention over Convolution Kernels Yinpeng Chen Xiyang Dai Mengchen Liu Dongdong Chen Lu Yuan Zicheng Liu 104 895 0 07 Dec 2019
PyTorch: An Imperative Style, High-Performance Deep Learning Library Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury ... Sasank Chilamkurthy Benoit Steiner Lu Fang Junjie Bai Soumith Chintala ODL 523 42,559 0 03 Dec 2019
On the Relationship between Self-Attention and Convolutional Layers Jean-Baptiste Cordonnier Andreas Loukas Martin Jaggi 116 535 0 08 Nov 2019
And the Bit Goes Down: Revisiting the Quantization of Neural Networks Pierre Stock Armand Joulin Rémi Gribonval Benjamin Graham Hervé Jégou MQ 82 149 0 12 Jul 2019
Efficient 8-Bit Quantization of Transformer Neural Machine Language Translation Model Aishwarya Bhandare Vamsi Sripathi Deepthi Karkada Vivek V. Menon Sun Choi Kushal Datta V. Saletore MQ 69 132 0 03 Jun 2019
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks Mingxing Tan Quoc V. Le 3DV MedIm 142 18,168 0 28 May 2019
Searching for MobileNetV3 Andrew G. Howard Mark Sandler Grace Chu Liang-Chieh Chen Bo Chen ... Yukun Zhu Ruoming Pang Vijay Vasudevan Quoc V. Le Hartwig Adam 359 6,799 0 06 May 2019
Generating Long Sequences with Sparse Transformers R. Child Scott Gray Alec Radford Ilya Sutskever 129 1,908 0 23 Apr 2019
Attention Augmented Convolutional Networks Irwan Bello Barret Zoph Ashish Vaswani Jonathon Shlens Quoc V. Le 137 1,015 0 22 Apr 2019
Selective Kernel Networks Xiang Li Wenhai Wang Xiaolin Hu Jian Yang 94 2,037 0 15 Mar 2019
MultiGrain: a unified image embedding for classes and instances Maxim Berman Hervé Jégou Andrea Vedaldi Iasonas Kokkinos Matthijs Douze 63 111 0 14 Feb 2019
Do ImageNet Classifiers Generalize to ImageNet? Benjamin Recht Rebecca Roelofs Ludwig Schmidt Vaishaal Shankar OOD SSeg VLM 118 1,726 0 13 Feb 2019
HAQ: Hardware-Aware Automated Quantization with Mixed Precision Kuan-Chieh Wang Zhijian Liu Chengyue Wu Ji Lin Song Han MQ 129 884 0 21 Nov 2018
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 95,114 0 11 Oct 2018
CBAM: Convolutional Block Attention Module Sanghyun Woo Jongchan Park Joon-Young Lee In So Kweon 224 16,582 0 17 Jul 2018
Self-Attention with Relative Position Representations Peter Shaw Jakob Uszkoreit Ashish Vaswani 177 2,295 0 06 Mar 2018
Image Transformer Niki Parmar Ashish Vaswani Jakob Uszkoreit Lukasz Kaiser Noam M. Shazeer Alexander Ku Dustin Tran ViT 138 1,684 0 15 Feb 2018
Non-local Neural Networks Xinyu Wang Ross B. Girshick Abhinav Gupta Kaiming He OffRL 289 8,916 0 21 Nov 2017
Squeeze-and-Excitation Networks Jie Hu Li Shen Samuel Albanie Gang Sun Enhua Wu 427 26,539 0 05 Sep 2017