Twins: Revisiting the Design of Spatial Attention in Vision Transformers

28 April 2021

Chunhua Shen

Papers citing "Twins: Revisiting the Design of Spatial Attention in Vision Transformers"

50 / 191 papers shown

Title
DropKey Bonan Li Yinhan Hu Xuecheng Nie Congying Han Xiangjian Jiang Tiande Guo Luoqi Liu 15 11 0 04 Aug 2022
MVSFormer: Multi-View Stereo by Learning Robust Image Features and Temperature-based Depth Chenjie Cao Xinlin Ren Yanwei Fu 31 46 0 04 Aug 2022
Local Perception-Aware Transformer for Aerial Tracking Changhong Fu Wei Peng Sihang Li Junjie Ye Ziang Cao 30 8 0 01 Aug 2022
TransNorm: Transformer Provides a Strong Spatial Normalization Mechanism for a Deep Segmentation Model Reza Azad Mohammad T. Al-Antary Moein Heidari Dorit Merhof ViT MedIm 28 72 0 27 Jul 2022
EleGANt: Exquisite and Locally Editable GAN for Makeup Transfer Chenyu Yang W. He Yingqing Xu Yang Gao DiffM 19 26 0 20 Jul 2022
Vision Transformers: From Semantic Segmentation to Dense Prediction Li Zhang Jiachen Lu Sixiao Zheng Xinxuan Zhao Xiatian Zhu Yanwei Fu Tao Xiang Jianfeng Feng Philip H. S. Torr ViT 24 7 0 19 Jul 2022
Next-ViT: Next Generation Vision Transformer for Efficient Deployment in Realistic Industrial Scenarios Jiashi Li Xin Xia W. Li Huixia Li Xing Wang Xuefeng Xiao Rui Wang Min Zheng Xin Pan ViT 17 149 0 12 Jul 2022
Dual Vision Transformer Ting Yao Yehao Li Yingwei Pan Yu Wang Xiaoping Zhang Tao Mei ViT 141 75 0 11 Jul 2022
Improving Semantic Segmentation in Transformers using Hierarchical Inter-Level Attention Gary Leung Jun Gao Fangyin Wei Sanja Fidler 21 3 0 05 Jul 2022
Learning Cross-Image Object Semantic Relation in Transformer for Few-Shot Fine-Grained Image Classification Bo-Wen Zhang Jiakang Yuan Baopu Li Tao Chen Jiayuan Fan Botian Shi ViT 21 31 0 02 Jul 2022
EATFormer: Improving Vision Transformer Inspired by Evolutionary Algorithm Jiangning Zhang Xiangtai Li Yabiao Wang Chengjie Wang Yibo Yang Yong Liu Dacheng Tao ViT 34 32 0 19 Jun 2022
SimA: Simple Softmax-free Attention for Vision Transformers Soroush Abbasi Koohpayegani Hamed Pirsiavash 16 25 0 17 Jun 2022
Which models are innately best at uncertainty estimation? Ido Galil Mohammed Dabbah Ran El-Yaniv UQCV 34 5 0 05 Jun 2022
Green Hierarchical Vision Transformer for Masked Image Modeling Lang Huang Shan You Mingkai Zheng Fei Wang Chao Qian T. Yamasaki 27 68 0 26 May 2022
Inception Transformer Chenyang Si Weihao Yu Pan Zhou Yichen Zhou Xinchao Wang Shuicheng Yan ViT 26 187 0 25 May 2022
ASSET: Autoregressive Semantic Scene Editing with Transformers at High Resolutions Difan Liu Sandesh Shetty Tobias Hinz Matthew Fisher Richard Y. Zhang Taesung Park E. Kalogerakis ViT 27 30 0 24 May 2022
Super Vision Transformer Mingbao Lin Mengzhao Chen Yu-xin Zhang Yunhang Shen Rongrong Ji Liujuan Cao ViT 40 20 0 23 May 2022
SelfReformer: Self-Refined Network with Transformer for Salient Object Detection Y. Yun Weisi Lin ViT 60 28 0 23 May 2022
Boosting Camouflaged Object Detection with Dual-Task Interactive Transformer Zheng Liu Zhili Zhang Wei Yu Wu 32 46 0 21 May 2022
MulT: An End-to-End Multitask Learning Transformer Deblina Bhattacharjee Tong Zhang Sabine Süsstrunk Mathieu Salzmann ViT 36 62 0 17 May 2022
Activating More Pixels in Image Super-Resolution Transformer Xiangyu Chen Xintao Wang Jiantao Zhou Yu Qiao Chao Dong ViT 64 601 0 09 May 2022
EdgeViTs: Competing Light-weight CNNs on Mobile Devices with Vision Transformers Junting Pan Adrian Bulat Fuwen Tan Xiatian Zhu L. Dudziak Hongsheng Li Georgios Tzimiropoulos Brais Martínez ViT 31 180 0 06 May 2022
Deeper Insights into the Robustness of ViTs towards Common Corruptions Rui Tian Zuxuan Wu Qi Dai Han Hu Yu-Gang Jiang ViT AAML 21 4 0 26 Apr 2022
$Cylin-Painting: Seamless {360\textdegree} Panoramic Image Outpainting and Beyond$ Cylin-Painting: Seamless {360\textdegree} Panoramic Image Outpainting and Beyond K. Liao Xiangyu Xu Chunyu Lin Wenqi Ren Yunchao Wei Yao Zhao 42 8 0 18 Apr 2022
VSA: Learning Varied-Size Window Attention in Vision Transformers Qiming Zhang Yufei Xu Jing Zhang Dacheng Tao 22 53 0 18 Apr 2022
3D Shuffle-Mixer: An Efficient Context-Aware Vision Learner of Transformer-MLP Paradigm for Dense Prediction in Medical Volume Jianye Pang Cheng Jiang Yihao Chen Jianbo Chang M. Feng Renzhi Wang Jianhua Yao ViT MedIm 28 11 0 14 Apr 2022
TopFormer: Token Pyramid Transformer for Mobile Semantic Segmentation Wenqiang Zhang Zilong Huang Guozhong Luo Tao Chen Xinggang Wang Wenyu Liu Gang Yu Chunhua Shen ViT 22 198 0 12 Apr 2022
Learning Local and Global Temporal Contexts for Video Semantic Segmentation Guolei Sun Yun Liu Henghui Ding Min Wu Luc Van Gool 30 32 0 07 Apr 2022
MixFormer: Mixing Features across Windows and Dimensions Qiang Chen Qiman Wu Jian Wang Qinghao Hu T. Hu Errui Ding Jian Cheng Jingdong Wang MDE ViT 28 101 0 06 Apr 2022
BatchFormerV2: Exploring Sample Relationships for Dense Representation Learning Zhi Hou Baosheng Yu Chaoyue Wang Yibing Zhan Dacheng Tao ViT 26 11 0 04 Apr 2022
Improving Vision Transformers by Revisiting High-frequency Components Jiawang Bai Liuliang Yuan Shutao Xia Shuicheng Yan Zhifeng Li Wei Liu ViT 16 90 0 03 Apr 2022
SepViT: Separable Vision Transformer Wei Li Xing Wang Xin Xia Jie Wu Jiashi Li Xuefeng Xiao Min Zheng Shiping Wen ViT 26 40 0 29 Mar 2022
Few-Shot Object Detection with Fully Cross-Transformer G. Han Jiawei Ma Shiyuan Huang Long Chen Shih-Fu Chang 26 130 0 28 Mar 2022
Transformers Meet Visual Learning Understanding: A Comprehensive Review Yuting Yang Licheng Jiao Xuantong Liu F. Liu Shuyuan Yang Zhixi Feng Xu Tang ViT MedIm 27 28 0 24 Mar 2022
Beyond Fixation: Dynamic Window Visual Transformer Pengzhen Ren Changlin Li Guangrun Wang Yun Xiao Qing Du Xiaodan Liang Qing Du Xiaodan Liang Xiaojun Chang ViT 28 32 0 24 Mar 2022
ScalableViT: Rethinking the Context-oriented Generalization of Vision Transformer Rui Yang Hailong Ma Jie Wu Yansong Tang Xuefeng Xiao Min Zheng Xiu Li ViT 19 53 0 21 Mar 2022
V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformer Runsheng Xu Hao Xiang Zhengzhong Tu Xin Xia Ming-Hsuan Yang Jiaqi Ma ViT 109 362 0 20 Mar 2022
HIPA: Hierarchical Patch Transformer for Single Image Super Resolution Qing Cai Yiming Qian Jinxing Li Junjie Lv Yee-Hong Yang Feng Wu Dafan Zhang 19 28 0 19 Mar 2022
Towards Data-Efficient Detection Transformers Wen Wang Jing Zhang Yang Cao Yongliang Shen Dacheng Tao ViT 23 59 0 17 Mar 2022
Smoothing Matters: Momentum Transformer for Domain Adaptive Semantic Segmentation Runfa Chen Yu Rong Shangmin Guo Jiaqi Han Gang Hua Tingyang Xu Wenbing Huang ViT 15 20 0 15 Mar 2022
Joint CNN and Transformer Network via weakly supervised Learning for efficient crowd counting Fusen Wang Kai Liu Fei Long Nong Sang Xiaofeng Xia J. Sang ViT 37 19 0 12 Mar 2022
CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformers Jiaming Zhang Huayao Liu Kailun Yang Xinxin Hu Ruiping Liu Rainer Stiefelhagen ViT 31 299 0 09 Mar 2022
CrowdFormer: Weakly-supervised Crowd counting with Improved Generalizability Siddharth Singh Savner Vivek Kanhangad ViT 19 31 0 07 Mar 2022
Multi-class Token Transformer for Weakly Supervised Semantic Segmentation Lian Xu Wanli Ouyang Bennamoun F. Boussaïd Dan Xu ViT 28 209 0 06 Mar 2022
Correlation-Aware Deep Tracking Fei Xie Chunyu Wang Guangting Wang Yue Cao Wankou Yang Wenjun Zeng VOT 29 119 0 03 Mar 2022
TransDARC: Transformer-based Driver Activity Recognition with Latent Space Feature Calibration Kunyu Peng Alina Roitberg Kailun Yang Jiaming Zhang Rainer Stiefelhagen ViT 31 32 0 02 Mar 2022
Delving Deep into One-Shot Skeleton-based Action Recognition with Diverse Occlusions Kunyu Peng Alina Roitberg Kailun Yang Jiaming Zhang Rainer Stiefelhagen ViT 21 28 0 23 Feb 2022
GroupViT: Semantic Segmentation Emerges from Text Supervision Jiarui Xu Shalini De Mello Sifei Liu Wonmin Byeon Thomas Breuel Jan Kautz Xinyu Wang ViT VLM 189 499 0 22 Feb 2022
Visual Attention Network Meng-Hao Guo Chengrou Lu Zheng-Ning Liu Ming-Ming Cheng Shiyong Hu ViT VLM 24 637 0 20 Feb 2022
How Do Vision Transformers Work? Namuk Park Songkuk Kim ViT 35 465 0 14 Feb 2022