Rethinking Spatial Dimensions of Vision Transformers

30 March 2021

Papers citing "Rethinking Spatial Dimensions of Vision Transformers"

50 / 307 papers shown

Title
Conv2Former: A Simple Transformer-Style ConvNet for Visual Recognition Qibin Hou Cheng Lu Mingg-Ming Cheng Jiashi Feng ViT 34 129 0 22 Nov 2022
Peeling the Onion: Hierarchical Reduction of Data Redundancy for Efficient Vision Transformer Training Zhenglun Kong Haoyu Ma Geng Yuan Mengshu Sun Yanyue Xie ... Tianlong Chen Xiaolong Ma Xiaohui Xie Zhangyang Wang Yanzhi Wang ViT 31 22 0 19 Nov 2022
Castling-ViT: Compressing Self-Attention via Switching Towards Linear-Angular Attention at Vision Transformer Inference Haoran You Yunyang Xiong Xiaoliang Dai Bichen Wu Peizhao Zhang Haoqi Fan Peter Vajda Yingyan Lin 35 32 0 18 Nov 2022
HeatViT: Hardware-Efficient Adaptive Token Pruning for Vision Transformers Peiyan Dong Mengshu Sun Alec Lu Yanyue Xie Li-Yu Daisy Liu ... Xin Meng ZeLin Li Xue Lin Zhenman Fang Yanzhi Wang ViT 31 59 0 15 Nov 2022
ParCNetV2: Oversized Kernel with Enhanced Attention Ruihan Xu Haokui Zhang Wenze Hu Shiliang Zhang Xiaoyu Wang ViT 30 6 0 14 Nov 2022
AU-Aware Vision Transformers for Biased Facial Expression Recognition Shuyi Mao Xinpeng Li Q. Wu Xiaojiang Peng ViT 36 2 0 12 Nov 2022
Token Transformer: Can class token help window-based transformer build better long-range interactions? Jia-ju Mao Yuan Chang Xuesong Yin 28 0 0 11 Nov 2022
A Comprehensive Survey of Transformers for Computer Vision Sonain Jamil Md. Jalil Piran Oh-Jin Kwon ViT 30 46 0 11 Nov 2022
ViTALiTy: Unifying Low-rank and Sparse Approximation for Vision Transformer Acceleration with a Linear Taylor Attention Jyotikrishna Dass Shang Wu Huihong Shi Chaojian Li Zhifan Ye Zhongfeng Wang Yingyan Lin 17 49 0 09 Nov 2022
Differentially Private CutMix for Split Learning with Vision Transformer Seungeun Oh Jihong Park Sihun Baek Hyelin Nam Praneeth Vepakomma Ramesh Raskar M. Bennis Seong-Lyun Kim FedML 19 16 0 28 Oct 2022
Grafting Vision Transformers Jong Sung Park Kumara Kahatapitiya Donghyun Kim Shivchander Sudalairaj Quanfu Fan Michael S. Ryoo ViT 26 2 0 28 Oct 2022
Automatic Diagnosis of Myocarditis Disease in Cardiac MRI Modality using Deep Transformers and Explainable Artificial Intelligence M. Jafari A. Shoeibi Navid Ghassemi Jónathan Heras Saiguang Ling ... Shuihua Wang R. Alizadehsani Juan M Gorriz U. Acharya Hamid Alinejad-Rokny MedIm 22 11 0 26 Oct 2022
Accumulated Trivial Attention Matters in Vision Transformers on Small Datasets Xiangyu Chen Qinghao Hu Kaidong Li Cuncong Zhong Guanghui Wang ViT 38 11 0 22 Oct 2022
Face Pyramid Vision Transformer Khawar Islam M. Zaheer Arif Mahmood ViT CVBM 24 4 0 21 Oct 2022
Boosting vision transformers for image retrieval Chull Hwan Song Jooyoung Yoon Shunghyun Choi Yannis Avrithis ViT 34 32 0 21 Oct 2022
Similarity of Neural Architectures using Adversarial Attack Transferability Jaehui Hwang Dongyoon Han Byeongho Heo Song Park Sanghyuk Chun Jong-Seok Lee AAML 32 1 0 20 Oct 2022
ViTCoD: Vision Transformer Acceleration via Dedicated Algorithm and Accelerator Co-Design Haoran You Zhanyi Sun Huihong Shi Zhongzhi Yu Yang Katie Zhao Yongan Zhang Chaojian Li Baopu Li Yingyan Lin ViT 25 76 0 18 Oct 2022
Scratching Visual Transformer's Back with Uniform Attention Nam Hyeon-Woo Kim Yu-Ji Byeongho Heo Doonyoon Han Seong Joon Oh Tae-Hyun Oh 353 23 0 16 Oct 2022
TokenMixup: Efficient Attention-guided Token-level Data Augmentation for Transformers Hyeong Kyu Choi Joonmyung Choi Hyunwoo J. Kim ViT 28 35 0 14 Oct 2022
Bridging the Gap Between Vision Transformers and Convolutional Neural Networks on Small Datasets Zhiying Lu Hongtao Xie Chuanbin Liu Yongdong Zhang ViT 25 57 0 12 Oct 2022
Fast-ParC: Capturing Position Aware Global Feature for ConvNets and ViTs Taojiannan Yang Haokui Zhang Wenze Hu C. L. P. Chen Xiaoyu Wang ViT 24 0 0 08 Oct 2022
The Lie Derivative for Measuring Learned Equivariance Nate Gruver Marc Finzi Micah Goldblum A. Wilson 18 34 0 06 Oct 2022
Natural Color Fool: Towards Boosting Black-box Unrestricted Attacks Shengming Yuan Qilong Zhang Lianli Gao Yaya Cheng Jingkuan Song AAML 24 42 0 05 Oct 2022
MOAT: Alternating Mobile Convolution and Attention Brings Strong Vision Models Chenglin Yang Siyuan Qiao Qihang Yu Xiaoding Yuan Yukun Zhu Alan Yuille Hartwig Adam Liang-Chieh Chen ViT MoE 36 58 0 04 Oct 2022
Expediting Large-Scale Vision Transformer for Dense Prediction without Fine-tuning Weicong Liang Yuhui Yuan Henghui Ding Xiao Luo Weihong Lin Ding Jia Zheng-Wei Zhang Chao Zhang Hanhua Hu 29 25 0 03 Oct 2022
Learning Hierarchical Image Segmentation For Recognition and By Recognition Tsung-Wei Ke Sangwoo Mo Stella X. Yu VLM 32 9 0 01 Oct 2022
MobileViTv3: Mobile-Friendly Vision Transformer with Simple and Effective Fusion of Local, Global and Input Features S. Wadekar Abhishek Chaurasia ViT 98 87 0 30 Sep 2022
Effective Vision Transformer Training: A Data-Centric Perspective Benjia Zhou Pichao Wang Jun Wan Yan-Ni Liang Fan Wang 26 5 0 29 Sep 2022
Strong Transferable Adversarial Attacks via Ensembled Asymptotically Normal Distribution Learning Zhengwei Fang Rui Wang Tao Huang L. Jing AAML 19 5 0 24 Sep 2022
Deep Convolutional Pooling Transformer for Deepfake Detection Tianyi Wang Harry Cheng Kam-pui Chow Liqiang Nie ViT 27 66 0 12 Sep 2022
Neural Feature Fusion Fields: 3D Distillation of Self-Supervised 2D Image Representations Vadim Tschernezki Iro Laina Diane Larlus Andrea Vedaldi 186 185 0 07 Sep 2022
ClusTR: Exploring Efficient Self-attention via Clustering for Vision Transformers Yutong Xie Jianpeng Zhang Yong-quan Xia Anton Van Den Hengel Qi Wu 30 6 0 28 Aug 2022
Transformer Vs. MLP-Mixer: Exponential Expressive Gap For NLP Problems D. Navon A. Bronstein MoE 38 0 0 17 Aug 2022
Convolutional Embedding Makes Hierarchical Vision Transformer Stronger Cong Wang Hongmin Xu Xiong Zhang Li Wang Zhitong Zheng Haifeng Liu ViT 17 20 0 27 Jul 2022
Locality Guidance for Improving Vision Transformers on Tiny Datasets Kehan Li Runyi Yu Zhennan Wang Li-ming Yuan Guoli Song Jie Chen ViT 29 43 0 20 Jul 2022
EleGANt: Exquisite and Locally Editable GAN for Makeup Transfer Chenyu Yang W. He Yingqing Xu Yang Gao DiffM 19 26 0 20 Jul 2022
AU-Supervised Convolutional Vision Transformers for Synthetic Facial Expression Recognition Shuyi Mao Xinpeng Li Jun-Cheng Chen Xiaojiang Peng ViT 19 0 0 20 Jul 2022
Vision Transformers: From Semantic Segmentation to Dense Prediction Li Zhang Jiachen Lu Sixiao Zheng Xinxuan Zhao Xiatian Zhu Yanwei Fu Tao Xiang Jianfeng Feng Philip H. S. Torr ViT 27 7 0 19 Jul 2022
Multi-manifold Attention for Vision Transformers D. Konstantinidis Ilias Papastratis K. Dimitropoulos P. Daras ViT 24 16 0 18 Jul 2022
On the Strong Correlation Between Model Invariance and Generalization Weijian Deng Stephen Gould Liang Zheng OOD 32 16 0 14 Jul 2022
Next-ViT: Next Generation Vision Transformer for Efficient Deployment in Realistic Industrial Scenarios Jiashi Li Xin Xia W. Li Huixia Li Xing Wang Xuefeng Xiao Rui Wang Min Zheng Xin Pan ViT 17 149 0 12 Jul 2022
Outpainting by Queries Kai Yao Penglei Gao Xi Yang Kaizhu Huang Jie Sun Rui Zhang ViT 33 13 0 12 Jul 2022
Dual Vision Transformer Ting Yao Yehao Li Yingwei Pan Yu Wang Xiaoping Zhang Tao Mei ViT 141 75 0 11 Jul 2022
Vision Transformers: State of the Art and Research Challenges Bo-Kai Ruan Hong-Han Shuai Wen-Huang Cheng ViT 27 17 0 07 Jul 2022
Visual Transformer Meets CutMix for Improved Accuracy, Communication Efficiency, and Data Privacy in Split Learning Sihun Baek Jihong Park Praneeth Vepakomma Ramesh Raskar M. Bennis Seong-Lyun Kim FedML 28 10 0 01 Jul 2022
Rethinking Query-Key Pairwise Interactions in Vision Transformers Cheng-rong Li Yangxin Liu 34 0 0 01 Jul 2022
Patch-level Representation Learning for Self-supervised Vision Transformers Sukmin Yun Hankook Lee Jaehyung Kim Jinwoo Shin ViT 22 64 0 16 Jun 2022
MobileOne: An Improved One millisecond Mobile Backbone Pavan Kumar Anasosalu Vasu J. Gabriel Jeff J. Zhu Oncel Tuzel Anurag Ranjan 30 154 0 08 Jun 2022
Separable Self-attention for Mobile Vision Transformers Sachin Mehta Mohammad Rastegari ViT MQ 23 251 0 06 Jun 2022
Which models are innately best at uncertainty estimation? Ido Galil Mohammed Dabbah Ran El-Yaniv UQCV 34 5 0 05 Jun 2022