SaiT: Sparse Vision Transformers through Adaptive Token Pruning

11 October 2022

Papers citing "SaiT: Sparse Vision Transformers through Adaptive Token Pruning"

32 / 32 papers shown

Title
Exploring Plain Vision Transformer Backbones for Object Detection Yanghao Li Hanzi Mao Ross B. Girshick Kaiming He ViT 104 816 0 30 Mar 2022
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 485 7,837 0 11 Nov 2021
Per-Pixel Classification is Not All You Need for Semantic Segmentation Bowen Cheng Alex Schwing Alexander Kirillov VLM ViT 214 1,554 0 13 Jul 2021
Post-Training Quantization for Vision Transformer Zhenhua Liu Yunhe Wang Kai Han Siwei Ma Wen Gao ViT MQ 108 343 0 27 Jun 2021
Chasing Sparsity in Vision Transformers: An End-to-End Exploration Tianlong Chen Yu Cheng Zhe Gan Lu Yuan Lei Zhang Zhangyang Wang ViT 70 222 0 08 Jun 2021
DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification Yongming Rao Wenliang Zhao Benlin Liu Jiwen Lu Jie Zhou Cho-Jui Hsieh ViT 109 709 0 03 Jun 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 743 6,139 0 29 Apr 2021
Twins: Revisiting the Design of Spatial Attention in Vision Transformers Xiangxiang Chu Zhi Tian Yuqing Wang Bo Zhang Haibing Ren Xiaolin K. Wei Huaxia Xia Chunhua Shen ViT 87 1,028 0 28 Apr 2021
All Tokens Matter: Token Labeling for Training Better Vision Transformers Zihang Jiang Qibin Hou Li-xin Yuan Daquan Zhou Yujun Shi Xiaojie Jin Anran Wang Jiashi Feng ViT 127 209 0 22 Apr 2021
Co-Scale Conv-Attentional Image Transformers Weijian Xu Yifan Xu Tyler A. Chang Zhuowen Tu ViT 59 377 0 13 Apr 2021
LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inference Ben Graham Alaaeldin El-Nouby Hugo Touvron Pierre Stock Armand Joulin Hervé Jégou Matthijs Douze ViT 97 794 0 02 Apr 2021
Going deeper with Image Transformers Hugo Touvron Matthieu Cord Alexandre Sablayrolles Gabriel Synnaeve Hervé Jégou ViT 170 1,022 0 31 Mar 2021
CvT: Introducing Convolutions to Vision Transformers Haiping Wu Bin Xiao Noel Codella Mengchen Liu Xiyang Dai Lu Yuan Lei Zhang ViT 162 1,921 0 29 Mar 2021
CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification Chun-Fu Chen Quanfu Fan Yikang Shen ViT 71 1,486 0 27 Mar 2021
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows Ze Liu Yutong Lin Yue Cao Han Hu Yixuan Wei Zheng Zhang Stephen Lin B. Guo ViT 472 21,656 0 25 Mar 2021
BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search Changlin Li Tao Tang Guangrun Wang Jiefeng Peng Bing Wang Xiaodan Liang Xiaojun Chang ViT 139 107 0 23 Mar 2021
Perceiver: General Perception with Iterative Attention Andrew Jaegle Felix Gimeno Andrew Brock Andrew Zisserman Oriol Vinyals João Carreira VLM ViT MDE 212 1,026 0 04 Mar 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 545 3,742 0 24 Feb 2021
Centroid Transformers: Learning to Abstract with Attention Lemeng Wu Xingchao Liu Qiang Liu 3DPC 94 29 0 17 Feb 2021
High-Performance Large-Scale Image Recognition Without Normalization Andrew Brock Soham De Samuel L. Smith Karen Simonyan VLM 295 523 0 11 Feb 2021
Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet Li-xin Yuan Yunpeng Chen Tao Wang Weihao Yu Yujun Shi Zihang Jiang Francis E. H. Tay Jiashi Feng Shuicheng Yan ViT 155 1,946 0 28 Jan 2021
Bottleneck Transformers for Visual Recognition A. Srinivas Nayeon Lee Niki Parmar Jonathon Shlens Pieter Abbeel Ashish Vaswani SLR 362 995 0 27 Jan 2021
Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers Sixiao Zheng Jiachen Lu Hengshuang Zhao Xiatian Zhu Zekun Luo ... Yanwei Fu Jianfeng Feng Tao Xiang Philip Torr Li Zhang ViT 198 2,912 0 31 Dec 2020
Training data-efficient image transformers & distillation through attention Hugo Touvron Matthieu Cord Matthijs Douze Francisco Massa Alexandre Sablayrolles Hervé Jégou ViT 393 6,813 0 23 Dec 2020
End-to-End Object Detection with Transformers Nicolas Carion Francisco Massa Gabriel Synnaeve Nicolas Usunier Alexander Kirillov Sergey Zagoruyko ViT 3DV PINN 460 13,153 0 26 May 2020
Designing Network Design Spaces Ilija Radosavovic Raj Prateek Kosaraju Ross B. Girshick Kaiming He Piotr Dollár GNN 107 1,697 0 30 Mar 2020
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks Mingxing Tan Quoc V. Le 3DV MedIm 197 18,224 0 28 May 2019
Revisiting Unreasonable Effectiveness of Data in Deep Learning Era Chen Sun Abhinav Shrivastava Saurabh Singh Abhinav Gupta VLM 212 2,411 0 10 Jul 2017
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.3K 194,641 0 10 Dec 2015
Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding Song Han Huizi Mao W. Dally 3DGS 263 8,864 0 01 Oct 2015
Distilling the Knowledge in a Neural Network Geoffrey E. Hinton Oriol Vinyals J. Dean FedML 367 19,764 0 09 Mar 2015
Compressing Deep Convolutional Networks using Vector Quantization Yunchao Gong Liu Liu Ming Yang Lubomir D. Bourdev MQ 179 1,171 0 18 Dec 2014