Conditional Positional Encodings for Vision Transformers

22 February 2021

Chunhua Shen

Papers citing "Conditional Positional Encodings for Vision Transformers"

50 / 124 papers shown

Title
Beyond Attentive Tokens: Incorporating Token Importance and Diversity for Efficient Vision Transformers Sifan Long Z. Zhao Jimin Pi Sheng-sheng Wang Jingdong Wang 22 29 0 21 Nov 2022
Grafting Vision Transformers Jong Sung Park Kumara Kahatapitiya Donghyun Kim Shivchander Sudalairaj Quanfu Fan Michael S. Ryoo ViT 26 2 0 28 Oct 2022
Clinically-Inspired Multi-Agent Transformers for Disease Trajectory Forecasting from Multimodal Data Huy Hoang Nguyen Matthew B. Blaschko S. Saarakkala A. Tiulpin MedIm AI4CE 48 15 0 25 Oct 2022
Boosting vision transformers for image retrieval Chull Hwan Song Jooyoung Yoon Shunghyun Choi Yannis Avrithis ViT 34 32 0 21 Oct 2022
Transformers Learn Shortcuts to Automata Bingbin Liu Jordan T. Ash Surbhi Goel A. Krishnamurthy Cyril Zhang OffRL LRM 46 156 0 19 Oct 2022
Sequence and Circle: Exploring the Relationship Between Patches Zhengyang Yu Jochen Triesch ViT 25 0 0 18 Oct 2022
TokenMixup: Efficient Attention-guided Token-level Data Augmentation for Transformers Hyeong Kyu Choi Joonmyung Choi Hyunwoo J. Kim ViT 28 35 0 14 Oct 2022
Bridging the Gap Between Vision Transformers and Convolutional Neural Networks on Small Datasets Zhiying Lu Hongtao Xie Chuanbin Liu Yongdong Zhang ViT 25 57 0 12 Oct 2022
MobileViTv3: Mobile-Friendly Vision Transformer with Simple and Effective Fusion of Local, Global and Input Features S. Wadekar Abhishek Chaurasia ViT 100 87 0 30 Sep 2022
Dense-TNT: Efficient Vehicle Type Classification Neural Network Using Satellite Imagery Ruikang Luo Yaofeng Song Haiying Zhao Yicheng Zhang Yi Zhang Nanbin Zhao Liping Huang Rong Su ViT 16 11 0 27 Sep 2022
Improved Image Classification with Token Fusion Keong-Hun Choi Jin-Woo Kim Yaolong Wang J. Ha ViT 19 0 0 19 Aug 2022
Flow-Guided Transformer for Video Inpainting Kaiwen Zhang Jingjing Fu Dong Liu ViT 35 68 0 14 Aug 2022
PointConvFormer: Revenge of the Point-based Convolution Wenxuan Wu Li Fuxin Qi Shan 3DPC 25 30 0 04 Aug 2022
MVSFormer: Multi-View Stereo by Learning Robust Image Features and Temperature-based Depth Chenjie Cao Xinlin Ren Yanwei Fu 31 46 0 04 Aug 2022
Multi-branch Cascaded Swin Transformers with Attention to k-space Sampling Pattern for Accelerated MRI Reconstruction Mevan Ekanayake Kamlesh Pawar Mehrtash Harandi Gary Egan Zhaolin Chen ViT MedIm 24 7 0 18 Jul 2022
Defect Transformer: An Efficient Hybrid Transformer Architecture for Surface Defect Detection Junpu Wang Guili Xu Fuju Yan Jinjin Wang Zhengsheng Wang ViT MedIm 26 66 0 17 Jul 2022
EATFormer: Improving Vision Transformer Inspired by Evolutionary Algorithm Jiangning Zhang Xiangtai Li Yabiao Wang Chengjie Wang Yibo Yang Yong Liu Dacheng Tao ViT 34 32 0 19 Jun 2022
SP-ViT: Learning 2D Spatial Priors for Vision Transformers Yuxuan Zhou Wangmeng Xiang Chong Li Biao Wang Xihan Wei Lei Zhang M. Keuper Xia Hua ViT 34 15 0 15 Jun 2022
HiViT: Hierarchical Vision Transformer Meets Masked Image Modeling Xiaosong Zhang Yunjie Tian Wei Huang QiXiang Ye Qi Dai Lingxi Xie Qi Tian 64 26 0 30 May 2022
ASSET: Autoregressive Semantic Scene Editing with Transformers at High Resolutions Difan Liu Sandesh Shetty Tobias Hinz Matthew Fisher Richard Y. Zhang Taesung Park E. Kalogerakis ViT 27 30 0 24 May 2022
Super Vision Transformer Mingbao Lin Mengzhao Chen Yu-xin Zhang Yunhang Shen Rongrong Ji Liujuan Cao ViT 46 20 0 23 May 2022
ConvMAE: Masked Convolution Meets Masked Autoencoders Peng Gao Teli Ma Hongsheng Li Ziyi Lin Jifeng Dai Yu Qiao ViT 19 121 0 08 May 2022
EdgeViTs: Competing Light-weight CNNs on Mobile Devices with Vision Transformers Junting Pan Adrian Bulat Fuwen Tan Xiatian Zhu L. Dudziak Hongsheng Li Georgios Tzimiropoulos Brais Martínez ViT 31 181 0 06 May 2022
Deeper Insights into the Robustness of ViTs towards Common Corruptions Rui Tian Zuxuan Wu Qi Dai Han Hu Yu-Gang Jiang ViT AAML 21 4 0 26 Apr 2022
VSA: Learning Varied-Size Window Attention in Vision Transformers Qiming Zhang Yufei Xu Jing Zhang Dacheng Tao 22 53 0 18 Apr 2022
DaViT: Dual Attention Vision Transformers Mingyu Ding Bin Xiao Noel Codella Ping Luo Jingdong Wang Lu Yuan ViT 48 240 0 07 Apr 2022
Multi-scale Context-aware Network with Transformer for Gait Recognition Duo-Lin Zhu Xiaohui Huang Xinggang Wang Bo Yang Botao He Wenyu Liu Bin Feng ViT CVBM 22 15 0 07 Apr 2022
Dynamic Focus-aware Positional Queries for Semantic Segmentation Haoyu He Jianfei Cai Zizheng Pan Jing Liu Jing Zhang Dacheng Tao Bohan Zhuang 34 17 0 04 Apr 2022
SepViT: Separable Vision Transformer Wei Li Xing Wang Xin Xia Jie Wu Jiashi Li Xuefeng Xiao Min Zheng Shiping Wen ViT 26 40 0 29 Mar 2022
Transformers Meet Visual Learning Understanding: A Comprehensive Review Yuting Yang Licheng Jiao Xuantong Liu F. Liu Shuyuan Yang Zhixi Feng Xu Tang ViT MedIm 27 28 0 24 Mar 2022
Beyond Fixation: Dynamic Window Visual Transformer Pengzhen Ren Changlin Li Guangrun Wang Yun Xiao Qing Du Xiaodan Liang Qing Du Xiaodan Liang Xiaojun Chang ViT 28 32 0 24 Mar 2022
Training-free Transformer Architecture Search Qinqin Zhou Kekai Sheng Xiawu Zheng Ke Li Xing Sun Yonghong Tian Jie Chen Rongrong Ji ViT 34 46 0 23 Mar 2022
ScalableViT: Rethinking the Context-oriented Generalization of Vision Transformer Rui Yang Hailong Ma Jie Wu Yansong Tang Xuefeng Xiao Min Zheng Xiu Li ViT 19 53 0 21 Mar 2022
HIPA: Hierarchical Patch Transformer for Single Image Super Resolution Qing Cai Yiming Qian Jinxing Li Junjie Lv Yee-Hong Yang Feng Wu Dafan Zhang 22 28 0 19 Mar 2022
XYLayoutLM: Towards Layout-Aware Multimodal Networks For Visually-Rich Document Understanding Zhangxuan Gu Changhua Meng Ke Wang Jun Lan Weiqiang Wang Ming Gu Liqing Zhang 37 76 0 14 Mar 2022
Scaling Up Your Kernels to 31x31: Revisiting Large Kernel Design in CNNs Xiaohan Ding Xinming Zhang Yi Zhou Jungong Han Guiguang Ding Jian Sun VLM 49 528 0 13 Mar 2022
Dynamic Group Transformer: A General Vision Transformer Backbone with Dynamic Group Attention Kai Liu Tianyi Wu Cong Liu Guodong Guo ViT 41 17 0 08 Mar 2022
Multi-class Token Transformer for Weakly Supervised Semantic Segmentation Lian Xu Wanli Ouyang Bennamoun F. Boussaïd Dan Xu ViT 31 209 0 06 Mar 2022
Correlation-Aware Deep Tracking Fei Xie Chunyu Wang Guangting Wang Yue Cao Wankou Yang Wenjun Zeng VOT 29 119 0 03 Mar 2022
Classification of Long Sequential Data using Circular Dilated Convolutional Neural Networks Lei Cheng Ruslan Khalitov Tong Yu Zhirong Yang 25 32 0 06 Jan 2022
Pale Transformer: A General Vision Transformer Backbone with Pale-Shaped Attention Sitong Wu Tianyi Wu Hao Hao Tan G. Guo ViT 31 70 0 28 Dec 2021
SeMask: Semantically Masked Transformers for Semantic Segmentation Jitesh Jain Anukriti Singh Nikita Orlov Zilong Huang Jiachen Li Steven Walton Humphrey Shi ViT 29 92 0 23 Dec 2021
Lite Vision Transformer with Enhanced Self-Attention Chenglin Yang Yilin Wang Jianming Zhang He Zhang Zijun Wei Zhe-nan Lin Alan Yuille ViT 21 112 0 20 Dec 2021
FaceFormer: Speech-Driven 3D Facial Animation with Transformers Yingruo Fan Zhaojiang Lin Jun Saito Wenping Wang Taku Komura CVBM 43 195 0 10 Dec 2021
DualFormer: Local-Global Stratified Transformer for Efficient Video Recognition Yuxuan Liang Pan Zhou Roger Zimmermann Shuicheng Yan ViT 29 21 0 09 Dec 2021
A Survey of Visual Transformers Yang Liu Yao Zhang Yixin Wang Feng Hou Jin Yuan Jiang Tian Yang Zhang Zhongchao Shi Jianping Fan Zhiqiang He 3DGS ViT 77 330 0 11 Nov 2021
Blending Anti-Aliasing into Vision Transformer Shengju Qian Hao Shao Yi Zhu Mu Li Jiaya Jia 26 20 0 28 Oct 2021
3D Object Tracking with Transformer Yubo Cui Zheng Fang Jiayao Shan Zuoxu Gu Sifan Zhou ViT 3DPC 21 59 0 28 Oct 2021
SOFT: Softmax-free Transformer with Linear Complexity Jiachen Lu Jinghan Yao Junge Zhang Martin Danelljan Hang Xu Weiguo Gao Chunjing Xu Thomas B. Schon Li Zhang 18 161 0 22 Oct 2021
Study of positional encoding approaches for Audio Spectrogram Transformers L. Pepino Pablo Riera Luciana Ferrer ViT 28 6 0 13 Oct 2021