Rethinking Spatial Dimensions of Vision Transformers

30 March 2021

Papers citing "Rethinking Spatial Dimensions of Vision Transformers"

50 / 307 papers shown

Title
A Fast Knowledge Distillation Framework for Visual Recognition Zhiqiang Shen Eric P. Xing VLM 14 45 0 02 Dec 2021
Searching the Search Space of Vision Transformer Minghao Chen Kan Wu Bolin Ni Houwen Peng Bei Liu Jianlong Fu Hongyang Chao Haibin Ling ViT 27 52 0 29 Nov 2021
SWAT: Spatial Structure Within and Among Tokens Kumara Kahatapitiya Michael S. Ryoo 25 6 0 26 Nov 2021
Self-slimmed Vision Transformer Zhuofan Zong Kunchang Li Guanglu Song Yali Wang Yu Qiao B. Leng Yu Liu ViT 21 30 0 24 Nov 2021
An Image Patch is a Wave: Phase-Aware Vision MLP Yehui Tang Kai Han Jianyuan Guo Chang Xu Yanxi Li Chao Xu Yunhe Wang 24 133 0 24 Nov 2021
Efficient Video Transformers with Spatial-Temporal Token Selection Junke Wang Xitong Yang Hengduo Li Li Liu Zuxuan Wu Yu-Gang Jiang ViT 21 63 0 23 Nov 2021
Semi-Supervised Vision Transformers Zejia Weng Xitong Yang Ang Li Zuxuan Wu Yu-Gang Jiang ViT 14 40 0 22 Nov 2021
Rethinking Query, Key, and Value Embedding in Vision Transformer under Tiny Model Constraints Jaesin Ahn Jiuk Hong Jeongwoo Ju Heechul Jung ViT 32 3 0 19 Nov 2021
TransMix: Attend to Mix for Vision Transformers Jieneng Chen Shuyang Sun Ju He Philip H. S. Torr Alan Yuille S. Bai ViT 28 103 0 18 Nov 2021
INTERN: A New Learning Paradigm Towards General Vision Jing Shao Siyu Chen Yangguang Li Kun Wang Zhen-fei Yin ... F. Yu Junjie Yan Dahua Lin Xiaogang Wang Yu Qiao 16 34 0 16 Nov 2021
A Survey of Visual Transformers Yang Liu Yao Zhang Yixin Wang Feng Hou Jin Yuan Jiang Tian Yang Zhang Zhongchao Shi Jianping Fan Zhiqiang He 3DGS ViT 77 330 0 11 Nov 2021
Sliced Recursive Transformer Zhiqiang Shen Zechun Liu Eric P. Xing ViT 17 27 0 09 Nov 2021
MVT: Multi-view Vision Transformer for 3D Object Recognition Shuo Chen Tan Yu Ping Li ViT 37 43 0 25 Oct 2021
Adaptive Multi-view and Temporal Fusing Transformer for 3D Human Pose Estimation Hui Shuai Lele Wu Qingshan Liu ViT 17 44 0 11 Oct 2021
Measure Twice, Cut Once: Quantifying Bias and Fairness in Deep Neural Networks Cody Blakeney G. Atkinson Nathaniel Huish Yan Yan V. Metsis Ziliang Zong 11 3 0 08 Oct 2021
ViDT: An Efficient and Effective Fully Transformer-based Object Detector Hwanjun Song Deqing Sun Sanghyuk Chun Varun Jampani Dongyoon Han Byeongho Heo Wonjae Kim Ming-Hsuan Yang 87 76 0 08 Oct 2021
Token Pooling in Vision Transformers D. Marin Jen-Hao Rick Chang Anurag Ranjan Anish K. Prabhu Mohammad Rastegari Oncel Tuzel ViT 76 66 0 08 Oct 2021
Ripple Attention for Visual Perception with Sub-quadratic Complexity Lin Zheng Huijie Pan Lingpeng Kong 26 3 0 06 Oct 2021
MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer Sachin Mehta Mohammad Rastegari ViT 218 1,213 0 05 Oct 2021
UFO-ViT: High Performance Linear Vision Transformer without Softmax Jeonggeun Song ViT 114 20 0 29 Sep 2021
Towards Transferable Adversarial Attacks on Vision Transformers Zhipeng Wei Jingjing Chen Micah Goldblum Zuxuan Wu Tom Goldstein Yu-Gang Jiang ViT AAML 24 111 0 09 Sep 2021
Scaled ReLU Matters for Training Vision Transformers Pichao Wang Xue Wang Haowen Luo Jingkai Zhou Zhipeng Zhou Fan Wang Hao Li R. L. Jin 19 41 0 08 Sep 2021
Searching for Efficient Multi-Stage Vision Transformers Yi-Lun Liao S. Karaman Vivienne Sze ViT 16 19 0 01 Sep 2021
StyleAugment: Learning Texture De-biased Representations by Style Augmentation without Pre-defined Textures Sanghyuk Chun Song Park 11 6 0 24 Aug 2021
Causal Attention for Unbiased Visual Recognition Tan Wang Chan Zhou Qianru Sun Hanwang Zhang OOD CML 32 108 0 19 Aug 2021
Polyp-PVT: Polyp Segmentation with Pyramid Vision Transformers B. Dong Wenhai Wang Deng-Ping Fan Jinpeng Li Huazhu Fu Ling Shao ViT MedIm 31 314 0 16 Aug 2021
Evo-ViT: Slow-Fast Token Evolution for Dynamic Vision Transformer Yifan Xu Zhijie Zhang Mengdan Zhang Kekai Sheng Ke Li Weiming Dong Liqing Zhang Changsheng Xu Xing Sun ViT 32 201 0 03 Aug 2021
S $^2$ -MLPv2: Improved Spatial-Shift MLP Architecture for Vision Tan Yu Xu Li Yunfeng Cai Mingming Sun Ping Li 39 50 0 02 Aug 2021
HAT: Hierarchical Aggregation Transformers for Person Re-identification Guowen Zhang Pingping Zhang Jinqing Qi Huchuan Lu ViT 20 115 0 13 Jul 2021
Visual Parser: Representing Part-whole Hierarchies with Transformers Shuyang Sun Xiaoyu Yue S. Bai Philip H. S. Torr 50 27 0 13 Jul 2021
LANA: Latency Aware Network Acceleration Pavlo Molchanov Jimmy Hall Hongxu Yin Jan Kautz Nicolò Fusi Arash Vahdat 25 11 0 12 Jul 2021
Vision Xformers: Efficient Attention for Image Classification Pranav Jeevan Amit Sethi ViT 22 13 0 05 Jul 2021
What Makes for Hierarchical Vision Transformer? Yuxin Fang Xinggang Wang Rui Wu Wenyu Liu ViT 18 9 0 05 Jul 2021
CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windows Xiaoyi Dong Jianmin Bao Dongdong Chen Weiming Zhang Nenghai Yu Lu Yuan Dong Chen B. Guo ViT 25 953 0 01 Jul 2021
Rethinking Token-Mixing MLP for MLP-based Vision Backbone Tan Yu Xu Li Yunfeng Cai Mingming Sun Ping Li 45 26 0 28 Jun 2021
VOLO: Vision Outlooker for Visual Recognition Li-xin Yuan Qibin Hou Zihang Jiang Jiashi Feng Shuicheng Yan ViT 52 313 0 24 Jun 2021
P2T: Pyramid Pooling Transformer for Scene Understanding Yu-Huan Wu Yun-Hai Liu Xin Zhan Mingg-Ming Cheng ViT 29 219 0 22 Jun 2021
S $^2$ -MLP: Spatial-Shift MLP Architecture for Vision Tan Yu Xu Li Yunfeng Cai Mingming Sun Ping Li 41 186 0 14 Jun 2021
MlTr: Multi-label Classification with Transformer Xingyi Cheng Hezheng Lin Xiangyu Wu Fan Yang Dong Shen Zhongyuan Wang Nian Shi Honglin Liu ViT 15 48 0 11 Jun 2021
Transformed CNNs: recasting pre-trained convolutional layers with self-attention Stéphane dÁscoli Levent Sagun Giulio Biroli Ari S. Morcos ViT 13 6 0 10 Jun 2021
On the Connection between Local Attention and Dynamic Depth-wise Convolution Qi Han Zejia Fan Qi Dai Lei-huan Sun Ming-Ming Cheng Jiaying Liu Jingdong Wang ViT 24 105 0 08 Jun 2021
ViTAE: Vision Transformer Advanced by Exploring Intrinsic Inductive Bias Yufei Xu Qiming Zhang Jing Zhang Dacheng Tao ViT 65 329 0 07 Jun 2021
Vision Transformers with Hierarchical Attention Yun-Hai Liu Yu-Huan Wu Guolei Sun Le Zhang Ajad Chhatkuli Luc Van Gool ViT 38 32 0 06 Jun 2021
Uformer: A General U-Shaped Transformer for Image Restoration Zhendong Wang Xiaodong Cun Jianmin Bao Wengang Zhou Jianzhuang Liu Houqiang Li ViT 48 1,368 0 06 Jun 2021
RegionViT: Regional-to-Local Attention for Vision Transformers Chun-Fu Chen Rameswar Panda Quanfu Fan ViT 16 194 0 04 Jun 2021
Container: Context Aggregation Network Peng Gao Jiasen Lu Hongsheng Li Roozbeh Mottaghi Aniruddha Kembhavi ViT 17 69 0 02 Jun 2021
Less is More: Pay Less Attention in Vision Transformers Zizheng Pan Bohan Zhuang Haoyu He Jing Liu Jianfei Cai ViT 24 82 0 29 May 2021
KVT: k-NN Attention for Boosting Vision Transformers Pichao Wang Xue Wang F. Wang Ming Lin Shuning Chang Hao Li R. L. Jin ViT 51 105 0 28 May 2021
Towards Robust Vision Transformer Xiaofeng Mao Gege Qi YueFeng Chen Xiaodan Li Ranjie Duan Shaokai Ye Yuan He Hui Xue ViT 23 186 0 17 May 2021
Do You Even Need Attention? A Stack of Feed-Forward Layers Does Surprisingly Well on ImageNet Luke Melas-Kyriazi ViT 9 101 0 06 May 2021