Transformer in Transformer

27 February 2021

Papers citing "Transformer in Transformer"

50 / 553 papers shown

Title
IA-RED $^2$ : Interpretability-Aware Redundancy Reduction for Vision Transformers Bowen Pan Yikang Shen Yi Ding Zhangyang Wang Rogerio Feris A. Oliva VLM ViT 39 153 0 23 Jun 2021
Vision Permutator: A Permutable MLP-Like Architecture for Visual Recognition Qibin Hou Zihang Jiang Li-xin Yuan Mingg-Ming Cheng Shuicheng Yan Jiashi Feng ViT MLLM 24 205 0 23 Jun 2021
More than Encoder: Introducing Transformer Decoder to Upsample Yijiang Li Wentian Cai Ying Gao Chengming Li Xiping Hu ViT MedIm 35 51 0 20 Jun 2021
How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers Andreas Steiner Alexander Kolesnikov Xiaohua Zhai Ross Wightman Jakob Uszkoreit Lucas Beyer ViT 66 616 0 18 Jun 2021
XCiT: Cross-Covariance Image Transformers Alaaeldin El-Nouby Hugo Touvron Mathilde Caron Piotr Bojanowski Matthijs Douze ... Ivan Laptev Natalia Neverova Gabriel Synnaeve Jakob Verbeek Hervé Jégou ViT 42 499 0 17 Jun 2021
S $^2$ -MLP: Spatial-Shift MLP Architecture for Vision Tan Yu Xu Li Yunfeng Cai Mingming Sun Ping Li 41 186 0 14 Jun 2021
MlTr: Multi-label Classification with Transformer Xingyi Cheng Hezheng Lin Xiangyu Wu Fan Yang Dong Shen Zhongyuan Wang Nian Shi Honglin Liu ViT 20 48 0 11 Jun 2021
CAT: Cross Attention in Vision Transformer Hezheng Lin Xingyi Cheng Xiangyu Wu Fan Yang Dong Shen Zhongyuan Wang Qing Song Wei Yuan ViT 35 149 0 10 Jun 2021
TED-net: Convolution-free T2T Vision Transformer-based Encoder-decoder Dilation network for Low-dose CT Denoising Dayang Wang Zhan Wu Hengyong Yu ViT MedIm 19 53 0 08 Jun 2021
A Survey of Transformers Tianyang Lin Yuxin Wang Xiangyang Liu Xipeng Qiu ViT 53 1,088 0 08 Jun 2021
Chasing Sparsity in Vision Transformers: An End-to-End Exploration Tianlong Chen Yu Cheng Zhe Gan Lu Yuan Lei Zhang Zhangyang Wang ViT 18 216 0 08 Jun 2021
On the Connection between Local Attention and Dynamic Depth-wise Convolution Qi Han Zejia Fan Qi Dai Lei-huan Sun Ming-Ming Cheng Jiaying Liu Jingdong Wang ViT 29 105 0 08 Jun 2021
Fully Transformer Networks for Semantic Image Segmentation Sitong Wu Tianyi Wu Fangjian Lin Sheng Tian Guodong Guo ViT 34 39 0 08 Jun 2021
Reveal of Vision Transformers Robustness against Adversarial Attacks Ahmed Aldahdooh W. Hamidouche Olivier Déforges ViT 15 56 0 07 Jun 2021
Refiner: Refining Self-attention for Vision Transformers Daquan Zhou Yujun Shi Bingyi Kang Weihao Yu Zihang Jiang Yuan Li Xiaojie Jin Qibin Hou Jiashi Feng ViT 29 59 0 07 Jun 2021
Shuffle Transformer: Rethinking Spatial Shuffle for Vision Transformer Zilong Huang Youcheng Ben Guozhong Luo Pei Cheng Gang Yu Bin-Bin Fu ViT 19 182 0 07 Jun 2021
ViTAE: Vision Transformer Advanced by Exploring Intrinsic Inductive Bias Yufei Xu Qiming Zhang Jing Zhang Dacheng Tao ViT 65 329 0 07 Jun 2021
Vision Transformers with Hierarchical Attention Yun-Hai Liu Yu-Huan Wu Guolei Sun Le Zhang Ajad Chhatkuli Luc Van Gool ViT 38 32 0 06 Jun 2021
Patch Slimming for Efficient Vision Transformers Yehui Tang Kai Han Yunhe Wang Chang Xu Jianyuan Guo Chao Xu Dacheng Tao ViT 24 163 0 05 Jun 2021
RegionViT: Regional-to-Local Attention for Vision Transformers Chun-Fu Chen Yikang Shen Quanfu Fan ViT 21 194 0 04 Jun 2021
Glance-and-Gaze Vision Transformer Qihang Yu Yingda Xia Yutong Bai Yongyi Lu Alan Yuille Wei Shen ViT 30 74 0 04 Jun 2021
DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification Yongming Rao Wenliang Zhao Benlin Liu Jiwen Lu Jie Zhou Cho-Jui Hsieh ViT 34 667 0 03 Jun 2021
SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers Enze Xie Wenhai Wang Zhiding Yu Anima Anandkumar J. Álvarez Ping Luo ViT 50 4,836 0 31 May 2021
MSG-Transformer: Exchanging Local Spatial Information by Manipulating Messenger Tokens Jiemin Fang Lingxi Xie Xinggang Wang Xiaopeng Zhang Wenyu Liu Qi Tian ViT 18 73 0 31 May 2021
Analogous to Evolutionary Algorithm: Designing a Unified Sequence Model Jiangning Zhang Chao Xu Jian Li Wenzhou Chen Yabiao Wang Ying Tai Shuo Chen Chengjie Wang Feiyue Huang Yong Liu 35 22 0 31 May 2021
Not All Images are Worth 16x16 Words: Dynamic Transformers for Efficient Image Recognition Yulin Wang Rui Huang S. Song Zeyi Huang Gao Huang ViT 33 189 0 31 May 2021
Dual-stream Network for Visual Recognition Mingyuan Mao Renrui Zhang Honghui Zheng Peng Gao Teli Ma Yan Peng Errui Ding Baochang Zhang Shumin Han ViT 25 63 0 31 May 2021
Gaze Estimation using Transformer Yihua Cheng Feng Lu ViT 21 87 0 30 May 2021
ResT: An Efficient Transformer for Visual Recognition Qing-Long Zhang Yubin Yang ViT 29 229 0 28 May 2021
KVT: k-NN Attention for Boosting Vision Transformers Pichao Wang Xue Wang F. Wang Ming Lin Shuning Chang Hao Li R. L. Jin ViT 51 105 0 28 May 2021
Nested Hierarchical Transformer: Towards Accurate, Data-Efficient and Interpretable Visual Understanding Zizhao Zhang Han Zhang Long Zhao Ting Chen Sercan Ö. Arik Tomas Pfister ViT 22 169 0 26 May 2021
TransLoc3D : Point Cloud based Large-scale Place Recognition using Adaptive Receptive Fields Tianhan Xu Yuanchen Guo Zhiqiang Li Ge Yu Yu-Kun Lai Song-Hai Zhang 3DPC 29 51 0 25 May 2021
Towards Robust Vision Transformer Xiaofeng Mao Gege Qi YueFeng Chen Xiaodan Li Ranjie Duan Shaokai Ye Yuan He Hui Xue ViT 23 186 0 17 May 2021
Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation Hu Cao Yueyue Wang Jieneng Chen Dongsheng Jiang Xiaopeng Zhang Qi Tian Manning Wang ViT MedIm 48 2,772 0 12 May 2021
Beyond Self-attention: External Attention using Two Linear Layers for Visual Tasks Meng-Hao Guo Zheng-Ning Liu Tai-Jiang Mu Shimin Hu 28 472 0 05 May 2021
Attention for Image Registration (AiR): an unsupervised Transformer approach Zihao Wang H. Delingette ViT MedIm 25 7 0 05 May 2021
Twins: Revisiting the Design of Spatial Attention in Vision Transformers Xiangxiang Chu Zhi Tian Yuqing Wang Bo Zhang Haibing Ren Xiaolin K. Wei Huaxia Xia Chunhua Shen ViT 23 997 0 28 Apr 2021
Vision Transformers with Patch Diversification Chengyue Gong Dilin Wang Meng Li Vikas Chandra Qiang Liu ViT 45 62 0 26 Apr 2021
All Tokens Matter: Token Labeling for Training Better Vision Transformers Zihang Jiang Qibin Hou Li-xin Yuan Daquan Zhou Yujun Shi Xiaojie Jin Anran Wang Jiashi Feng ViT 25 203 0 22 Apr 2021
Vision Transformer Pruning Mingjian Zhu Yehui Tang Kai Han ViT 11 90 0 17 Apr 2021
Going deeper with Image Transformers Hugo Touvron Matthieu Cord Alexandre Sablayrolles Gabriel Synnaeve Hervé Jégou ViT 27 986 0 31 Mar 2021
CvT: Introducing Convolutions to Vision Transformers Haiping Wu Bin Xiao Noel Codella Mengchen Liu Xiyang Dai Lu Yuan Lei Zhang ViT 72 1,878 0 29 Mar 2021
CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification Chun-Fu Chen Quanfu Fan Yikang Shen ViT 28 1,424 0 27 Mar 2021
Face Transformer for Recognition Yaoyao Zhong Weihong Deng ViT 19 70 0 27 Mar 2021
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows Ze Liu Yutong Lin Yue Cao Han Hu Yixuan Wei Zheng-Wei Zhang Stephen Lin B. Guo ViT 148 20,710 0 25 Mar 2021
BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search Changlin Li Tao Tang Guangrun Wang Jiefeng Peng Bing Wang Xiaodan Liang Xiaojun Chang ViT 48 105 0 23 Mar 2021
Prioritized Architecture Sampling with Monto-Carlo Tree Search Xiu Su Tao Huang Yanxi Li Shan You Fei Wang Chao Qian Changshui Zhang Chang Xu 19 49 0 22 Mar 2021
UPANets: Learning from the Universal Pixel Attention Networks Ching-Hsun Tseng Shin-Jye Lee Jianxing Feng Shengzhong Mao Yuping Wu Jia-Yu Shang Mou-Chung Tseng Xiao-Jun Zeng 22 15 0 15 Mar 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 316 3,625 0 24 Feb 2021
Segmenting Transparent Object in the Wild with Transformer Enze Xie Wenjia Wang Wenhai Wang Pei Sun Hang Xu Ding Liang Ping Luo ViT 227 28 0 21 Jan 2021