Softmax-free Linear Transformers

Softmax-free Linear Transformers

5 July 2022

Junge Zhang

Tao Xiang

Li Zhang

Papers citing "Softmax-free Linear Transformers"

8 / 8 papers shown

Title
Breaking the Low-Rank Dilemma of Linear Attention Qihang Fan Huaibo Huang Ran He 47 1 0 12 Nov 2024
Is Attention Better Than Matrix Decomposition? Zhengyang Geng Meng-Hao Guo Hongxu Chen Xia Li Ke Wei Zhouchen Lin 62 137 0 09 Sep 2021
Visual Parser: Representing Part-whole Hierarchies with Transformers Shuyang Sun Xiaoyu Yue S. Bai Philip Torr 50 27 0 13 Jul 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 289 3,623 0 24 Feb 2021
LambdaNetworks: Modeling Long-Range Interactions Without Attention Irwan Bello 278 179 0 17 Feb 2021
Bottleneck Transformers for Visual Recognition A. Srinivas Nayeon Lee Niki Parmar Jonathon Shlens Pieter Abbeel Ashish Vaswani SLR 290 979 0 27 Jan 2021
Aggregated Residual Transformations for Deep Neural Networks Saining Xie Ross B. Girshick Piotr Dollár Z. Tu Kaiming He 297 10,220 0 16 Nov 2016
Semantic Understanding of Scenes through the ADE20K Dataset Bolei Zhou Hang Zhao Xavier Puig Tete Xiao Sanja Fidler Adela Barriuso Antonio Torralba SSeg 253 1,828 0 18 Aug 2016