Beyond Self-attention: External Attention using Two Linear Layers for Visual Tasks

5 May 2021

Papers citing "Beyond Self-attention: External Attention using Two Linear Layers for Visual Tasks"

40 / 40 papers shown

Title
From Large to Super-Tiny: End-to-End Optimization for Cost-Efficient LLMs Jiliang Ni Jiachen Pu Zhongyi Yang Kun Zhou Hui Wang Xiaoliang Xiao Dakui Wang Xin Li Jingfeng Luo Conggang Hu 37 0 0 18 Apr 2025
Quantum Complex-Valued Self-Attention Model Fu Chen Qinglin Zhao Li Feng Longfei Tang Yangbin Lin Haitao Huang MQ 59 0 0 24 Mar 2025
Threshold Attention Network for Semantic Segmentation of Remote Sensing Images Wei Long Yongjun Zhang Zhongwei Cui Yujie Xu Xuexue Zhang 79 11 0 14 Jan 2025
SegSTRONG-C: Segmenting Surgical Tools Robustly On Non-adversarial Generated Corruptions -- An EndoVis'24 Challenge Hao Ding Tuxun Lu Yuqian Zhang Ruixing Liang Hongchao Shu ... Bo Wang Marcos Fernández-Rodríguez Estevao Lima João L. Vilaça Mathias Unberath 63 4 0 16 Jul 2024
Attributes Grouping and Mining Hashing for Fine-Grained Image Retrieval Xin Lu Shikun Chen Yichao Cao Xin Zhou Xiaobo Lu 28 12 0 10 Nov 2023
Real-time Automatic M-mode Echocardiography Measurement with Panel Attention from Local-to-Global Pixels Ching-Hsun Tseng S. Chien Po-Shen Wang Shin-Jye Lee Wei-Huan Hu Bin Pu Xiaojun Zeng 27 1 0 15 Aug 2023
Detection of Pavement Cracks by Deep Learning Models of Transformer and UNet Yu Zhang Lin Zhang UQCV MedIm ViT 30 10 0 25 Apr 2023
Soft Error Reliability Analysis of Vision Transformers Xing-xiong Xue Cheng Liu Ying Wang Bing Yang Tao Luo L. Zhang Huawei Li Xiaowei Li 34 14 0 21 Feb 2023
Towards Lightweight Cross-domain Sequential Recommendation via External Attention-enhanced Graph Convolution Network Jinyu Zhang Huichuan Duan Lei Guo Liancheng Xu Xinhua Wang GNN 42 4 0 07 Feb 2023
Long Range Pooling for 3D Large-Scale Scene Understanding Xiang-Li Li Meng-Hao Guo Tai-Jiang Mu Ralph Robert Martin Shiyong Hu 3DV 3DPC 19 2 0 17 Jan 2023
Multi-Stage Spatio-Temporal Aggregation Transformer for Video Person Re-identification Ziyi Tang Ruimao Zhang Zhanglin Peng Jinrui Chen Liang Lin 30 18 0 02 Jan 2023
Completing point cloud from few points by Wasserstein GAN and Transformers Xianfeng Wu Jinhui Qian Qing Wei Xian-Chun Wu Xinyi Liu Luxin Hu Yanli Gong Zhongyuan Lai Libing Wu ViT 3DPC 9 0 0 23 Nov 2022
MALUNet: A Multi-Attention and Light-weight UNet for Skin Lesion Segmentation Jiacheng Ruan Suncheng Xiang Mingye Xie Ting Liu Yuzhuo Fu 23 132 0 03 Nov 2022
Interpretable CNN-Multilevel Attention Transformer for Rapid Recognition of Pneumonia from Chest X-Ray Images Shengchao Chen Sufen Ren Guanjun Wang Mengxing Huang Chenyang Xue ViT MedIm 47 16 0 29 Oct 2022
Delving into Masked Autoencoders for Multi-Label Thorax Disease Classification Junfei Xiao Yutong Bai Alan Yuille Zongwei Zhou MedIm ViT 37 59 0 23 Oct 2022
Valuing Vicinity: Memory attention framework for context-based semantic segmentation in histopathology Oliver Ester Fabian Horst C. Seibold J. Keyl Saskia Ting ... P. Ivanyi Viktor Grünwald J. Bräsen Jan Egger Jens Kleesiek 21 7 0 21 Oct 2022
EMA-VIO: Deep Visual-Inertial Odometry with External Memory Attention Zheming Tu Changhao Chen Xianfei Pan Ruochen Liu Jiarui Cui Jun Mao 53 16 0 18 Sep 2022
Efficient Attention-free Video Shift Transformers Adrian Bulat Brais Martínez Georgios Tzimiropoulos ViT 29 1 0 23 Aug 2022
Eliminating Gradient Conflict in Reference-based Line-Art Colorization Zekun Li Zhengyang Geng Zhao Kang Wenyu Chen Yibo Yang 21 35 0 13 Jul 2022
SimA: Simple Softmax-free Attention for Vision Transformers Soroush Abbasi Koohpayegani Hamed Pirsiavash 16 25 0 17 Jun 2022
Visual Attention Network Meng-Hao Guo Chengrou Lu Zheng-Ning Liu Ming-Ming Cheng Shiyong Hu ViT VLM 24 637 0 20 Feb 2022
Real-World Graph Convolution Networks (RW-GCNs) for Action Recognition in Smart Video Surveillance Justin Sanchez Christopher Neff Hamed Tabkhi GNN 30 9 0 15 Jan 2022
An Image Patch is a Wave: Phase-Aware Vision MLP Yehui Tang Kai Han Jianyuan Guo Chang Xu Yanxi Li Chao Xu Yunhe Wang 24 133 0 24 Nov 2021
Mixed Transformer U-Net For Medical Image Segmentation Hongyi Wang Shiao Xie Lanfen Lin Yutaro Iwamoto X. Han Yenwei Chen Ruofeng Tong ViT MedIm 19 244 0 08 Nov 2021
Are we ready for a new paradigm shift? A Survey on Visual Deep MLP Ruiyang Liu Hai-Tao Zheng Li Tao Dun Liang Haitao Zheng 85 97 0 07 Nov 2021
Sparse MLP for Image Recognition: Is Self-Attention Really Necessary? Chuanxin Tang Yucheng Zhao Guangting Wang Chong Luo Wenxuan Xie Wenjun Zeng MoE ViT 35 98 0 12 Sep 2021
Is Attention Better Than Matrix Decomposition? Zhengyang Geng Meng-Hao Guo Hongxu Chen Xia Li Ke Wei Zhouchen Lin 59 137 0 09 Sep 2021
Query2Label: A Simple Transformer Way to Multi-Label Classification Shilong Liu Lei Zhang Xiao Yang Hang Su Jun Zhu 21 187 0 22 Jul 2021
CycleMLP: A MLP-like Architecture for Dense Prediction Shoufa Chen Enze Xie Chongjian Ge Runjian Chen Ding Liang Ping Luo 21 231 0 21 Jul 2021
Rethinking Token-Mixing MLP for MLP-based Vision Backbone Tan Yu Xu Li Yunfeng Cai Mingming Sun Ping Li 45 26 0 28 Jun 2021
Vision Permutator: A Permutable MLP-Like Architecture for Visual Recognition Qibin Hou Zihang Jiang Li-xin Yuan Mingg-Ming Cheng Shuicheng Yan Jiashi Feng ViT MLLM 24 205 0 23 Jun 2021
ViTAE: Vision Transformer Advanced by Exploring Intrinsic Inductive Bias Yufei Xu Qiming Zhang Jing Zhang Dacheng Tao ViT 62 329 0 07 Jun 2021
Transformer in Transformer Kai Han An Xiao Enhua Wu Jianyuan Guo Chunjing Xu Yunhe Wang ViT 287 1,524 0 27 Feb 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 277 3,623 0 24 Feb 2021
TransReID: Transformer-based Object Re-Identification Shuting He Haowen Luo Pichao Wang F. Wang Hao Li Wei Jiang ViT 213 794 0 08 Feb 2021
Transformers in Vision: A Survey Salman Khan Muzammal Naseer Munawar Hayat Syed Waqas Zamir F. Khan M. Shah ViT 227 2,430 0 04 Jan 2021
Spatial Pyramid Based Graph Reasoning for Semantic Segmentation Xia Li Yibo Yang Qijie Zhao Tiancheng Shen Zhouchen Lin Hong Liu SSeg GNN 112 161 0 23 Mar 2020
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation C. Qi Hao Su Kaichun Mo Leonidas J. Guibas 3DH 3DPC 3DV PINN 222 14,103 0 02 Dec 2016
Aggregated Residual Transformations for Deep Neural Networks Saining Xie Ross B. Girshick Piotr Dollár Z. Tu Kaiming He 297 10,220 0 16 Nov 2016
Semantic Understanding of Scenes through the ADE20K Dataset Bolei Zhou Hang Zhao Xavier Puig Tete Xiao Sanja Fidler Adela Barriuso Antonio Torralba SSeg 253 1,828 0 18 Aug 2016