Rethinking and Improving Relative Position Encoding for Vision Transformer

29 July 2021

ArXiv (abs)PDF HTML Github (1759★)

Papers citing "Rethinking and Improving Relative Position Encoding for Vision Transformer"

50 / 168 papers shown

Title
DilateFormer: Multi-Scale Dilated Transformer for Visual Recognition Jiayu Jiao Yuyao Tang Kun-Li Channing Lin Yipeng Gao Jinhua Ma Yaowei Wang Wei-Shi Zheng MedIm ViT 98 155 0 03 Feb 2023
Variation-Aware Semantic Image Synthesis Mingle Xu Jaehwan Lee Sook Yoon Hyongsuk Kim D. Park 72 4 0 25 Jan 2023
Text to Point Cloud Localization with Relation-Enhanced Transformer Guangzhi Wang Hehe Fan Mohan S. Kankanhalli 3DPC 81 15 0 13 Jan 2023
Head-Free Lightweight Semantic Segmentation with Linear Transformer B. Dong Pichao Wang Fan Wang ViT 78 75 0 11 Jan 2023
Learning to Exploit Temporal Structure for Biomedical Vision-Language Processing Shruthi Bannur Stephanie L. Hyland Qianchu Liu Fernando Pérez-García Maximilian Ilse ... Maria T. A. Wetscherek M. Lungren A. Nori Javier Alvarez-Valle Ozan Oktay 87 127 0 11 Jan 2023
Position Embedding Needs an Independent Layer Normalization Runyi Yu Zhennan Wang Yinhuai Wang Kehan Li Yian Zhao Jian Zhang Guoli Song Jie Chen 98 1 0 10 Dec 2022
Gaussian Radar Transformer for Semantic Segmentation in Noisy Radar Data Matthias Zeller Jens Behley Michael Heidingsfeld C. Stachniss 95 24 0 07 Dec 2022
Relation-Aware Language-Graph Transformer for Question Answering Jinyoung Park Hyeong Kyu Choi Juyeon Ko Hyeon-ju Park Ji-Hoon Kim Jisu Jeong Kyungmin Kim Hyunwoo J. Kim KELM LMTD ViT 46 10 0 02 Dec 2022
ResFormer: Scaling ViTs with Multi-Resolution Training Rui Tian Zuxuan Wu Qiuju Dai Hang-Rui Hu Yu Qiao Yu-Gang Jiang ViT 95 35 0 01 Dec 2022
AirFormer: Predicting Nationwide Air Quality in China with Transformers Yuxuan Liang Yutong Xia Songyu Ke Yiwei Wang Qingsong Wen Junbo Zhang Yu Zheng Roger Zimmermann AI4TS AI4CE 69 118 0 29 Nov 2022
Beyond Ensemble Averages: Leveraging Climate Model Ensembles for Subseasonal Forecasting Elena Orlova Haokun Liu Raphael Rossellini B. Cash Rebecca Willett 105 3 0 29 Nov 2022
Meta Architecture for Point Cloud Analysis Haojia Lin Xiawu Zheng Lijiang Li Chia-Wen Lin Sha Wang Yan Wang Yonghong Tian Rongrong Ji 3DPC 72 48 0 26 Nov 2022
Language Conditioned Spatial Relation Reasoning for 3D Object Grounding Shizhe Chen Pierre-Louis Guhur Makarand Tapaswi Cordelia Schmid Ivan Laptev 99 88 0 17 Nov 2022
Hypergraph Transformer for Skeleton-based Action Recognition Yuxuan Zhou Zhi-Qi Cheng Chong Li Yanwen Fang Yifeng Geng Xuansong Xie Margret Keuper ViT 94 60 0 17 Nov 2022
Parameter-Efficient Transformer with Hybrid Axial-Attention for Medical Image Segmentation Yiyue Hu Lei Zhang Nan Mu Leijun Liu ViT MedIm 44 1 0 17 Nov 2022
MogaNet: Multi-order Gated Aggregation Network Siyuan Li Zedong Wang Zicheng Liu Cheng Tan Haitao Lin Di Wu Zhiyuan Chen Jiangbin Zheng Stan Z. Li 103 65 0 07 Nov 2022
Data Level Lottery Ticket Hypothesis for Vision Transformers Xuan Shen Zhenglun Kong Minghai Qin Peiyan Dong Geng Yuan Xin Meng Hao Tang Xiaolong Ma Yanzhi Wang 87 6 0 02 Nov 2022
Adversarial Pretraining of Self-Supervised Deep Networks: Past, Present and Future Guo-Jun Qi M. Shah SSL 78 8 0 23 Oct 2022
Sequence and Circle: Exploring the Relationship Between Patches Zhengyang Yu Jochen Triesch ViT 50 0 0 18 Oct 2022
Dense-TNT: Efficient Vehicle Type Classification Neural Network Using Satellite Imagery Ruikang Luo Yaofeng Song Haiying Zhao Yicheng Zhang Yi Zhang Nanbin Zhao Liping Huang Rong Su ViT 42 12 0 27 Sep 2022
PSAQ-ViT V2: Towards Accurate and General Data-Free Quantization for Vision Transformers Zhikai Li Mengjuan Chen Junrui Xiao Qingyi Gu ViT MQ 123 35 0 13 Sep 2022
FocusFormer: Focusing on What We Need via Architecture Sampler Jing Liu Jianfei Cai Bohan Zhuang 58 8 0 23 Aug 2022
SoMoFormer: Social-Aware Motion Transformer for Multi-Person Motion Prediction Xiaogang Peng Yaodi Shen Haoran Wang Binling Nie Yigang Wang Zizhao Wu ViT 59 7 0 19 Aug 2022
giMLPs: Gate with Inhibition Mechanism in MLPs Cheng Kang Jindich Prokop Lei Tong Huiyu Zhou Yong Hu Daneil Novak 33 0 0 01 Aug 2022
TinyViT: Fast Pretraining Distillation for Small Vision Transformers Kan Wu Jinnian Zhang Houwen Peng Mengchen Liu Bin Xiao Jianlong Fu Lu Yuan ViT 74 267 0 21 Jul 2022
Parameterization of Cross-Token Relations with Relative Positional Encoding for Vision MLP Zhicai Wang Y. Hao Xingyu Gao Hao Zhang Shuo Wang Tingting Mu Xiangnan He 70 8 0 15 Jul 2022
I-ViT: Integer-only Quantization for Efficient Vision Transformer Inference Zhikai Li Qingyi Gu MQ 125 106 0 04 Jul 2022
LargeKernel3D: Scaling up Kernels in 3D Sparse CNNs Yukang Chen Jianhui Liu Xinming Zhang Xiaojuan Qi Jiaya Jia 124 90 0 21 Jun 2022
Online Segmentation of LiDAR Sequences: Dataset and Algorithm Romain Loiseau Mathieu Aubry Loïc Landrieu 3DPC 93 15 0 16 Jun 2022
SP-ViT: Learning 2D Spatial Priors for Vision Transformers Yuxuan Zhou Wangmeng Xiang Chong Li Biao Wang Xihan Wei Lei Zhang Margret Keuper Xia Hua ViT 71 15 0 15 Jun 2022
Peripheral Vision Transformer Juhong Min Yucheng Zhao Chong Luo Minsu Cho ViT MDE 72 33 0 14 Jun 2022
Positional Label for Self-Supervised Vision Transformer Zhemin Zhang Xun Gong ViT MDE 59 6 0 10 Jun 2022
Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectives Jun Li Junyu Chen Yucheng Tang Ce Wang Bennett A. Landman S. K. Zhou ViT OOD MedIm 169 43 0 02 Jun 2022
Modeling Image Composition for Complex Scene Generation Zuopeng Yang Daqing Liu Chaoyue Wang J. Yang Dacheng Tao ViT 113 52 0 02 Jun 2022
Vision GNN: An Image is Worth Graph of Nodes Kai Han Yunhe Wang Jianyuan Guo Yehui Tang Enhua Wu GNN 3DH 116 371 0 01 Jun 2022
Efficient Self-supervised Vision Pretraining with Local Masked Reconstruction Jun Chen Ming Hu Boyang Albert Li Mohamed Elhoseiny 142 37 0 01 Jun 2022
Flexible Diffusion Modeling of Long Videos William Harvey Saeid Naderiparizi Vaden Masrani Christian D. Weilbach Frank Wood DiffM BDL VGen 234 298 0 23 May 2022
KERPLE: Kernelized Relative Positional Embedding for Length Extrapolation Ta-Chung Chi Ting-Han Fan Peter J. Ramadge Alexander I. Rudnicky 98 73 0 20 May 2022
BabyNet: Residual Transformer Module for Birth Weight Prediction on Fetal Ultrasound Video Szymon Płotka Michal K. Grzeszczyk R. Brawura-Biskupski-Samaha P. Gutaj M. Lipa Tomasz Trzciñski Arkadiusz Sitek 3DH MedIm 40 17 0 19 May 2022
MiniViT: Compressing Vision Transformers with Weight Multiplexing Jinnian Zhang Houwen Peng Kan Wu Mengchen Liu Bin Xiao Jianlong Fu Lu Yuan ViT 90 127 0 14 Apr 2022
DaViT: Dual Attention Vision Transformers Mingyu Ding Bin Xiao Noel Codella Ping Luo Jingdong Wang Lu Yuan ViT 164 253 0 07 Apr 2022
Stratified Transformer for 3D Point Cloud Segmentation Xin Lai Jianhui Liu Li Jiang Liwei Wang Hengshuang Zhao Shu Liu Xiaojuan Qi Jiaya Jia 3DPC ViT 121 276 0 28 Mar 2022
Visual Abductive Reasoning Chen Liang Wenguan Wang Tianfei Zhou Yi Yang LRM 83 40 0 26 Mar 2022
Transformers Meet Visual Learning Understanding: A Comprehensive Review Yuting Yang Licheng Jiao Xuantong Liu Fan Liu Shuyuan Yang Zhixi Feng Xu Tang ViT MedIm 116 28 0 24 Mar 2022
PETR: Position Embedding Transformation for Multi-View 3D Object Detection Yingfei Liu Tiancai Wang Xinming Zhang Jian Sun 3DPC 144 553 0 10 Mar 2022
Patch Similarity Aware Data-Free Quantization for Vision Transformers Zhikai Li Liping Ma Mengjuan Chen Junrui Xiao Qingyi Gu MQ ViT 113 46 0 04 Mar 2022
Multi-Tailed Vision Transformer for Efficient Inference Yunke Wang Bo Du Wenyuan Wang Chang Xu ViT 315 6 0 03 Mar 2022
Recent Advances in Vision Transformer: A Survey and Outlook of Recent Work Khawar Islam ViT 153 51 0 03 Mar 2022
A Unified Query-based Paradigm for Point Cloud Understanding Zetong Yang Li Jiang Yanan Sun Bernt Schiele Jiaya Jia 3DPC 100 39 0 02 Mar 2022
Hilbert Flattening: a Locality-Preserving Matrix Unfolding Method for Visual Discrimination Qingsong Zhao Shuguang Dou Zhipeng Zhou Yangguang Li Yin Wang Yu Qiao Cairong Zhao 26 3 0 21 Feb 2022