LambdaNetworks: Modeling Long-Range Interactions Without Attention

17 February 2021

Papers citing "LambdaNetworks: Modeling Long-Range Interactions Without Attention"

50 / 117 papers shown

Title
FMMformer: Efficient and Flexible Transformer via Decomposed Near-field and Far-field Attention T. Nguyen Vai Suliafu Stanley J. Osher Long Chen Bao Wang 26 35 0 05 Aug 2021
Contextual Transformer Networks for Visual Recognition Yehao Li Ting Yao Yingwei Pan Tao Mei ViT 22 468 0 26 Jul 2021
Visual Parser: Representing Part-whole Hierarchies with Transformers Shuyang Sun Xiaoyu Yue S. Bai Philip H. S. Torr 50 27 0 13 Jul 2021
LANA: Latency Aware Network Acceleration Pavlo Molchanov Jimmy Hall Hongxu Yin Jan Kautz Nicolò Fusi Arash Vahdat 25 11 0 12 Jul 2021
VOLO: Vision Outlooker for Visual Recognition Li-xin Yuan Qibin Hou Zihang Jiang Jiashi Feng Shuicheng Yan ViT 52 313 0 24 Jun 2021
XCiT: Cross-Covariance Image Transformers Alaaeldin El-Nouby Hugo Touvron Mathilde Caron Piotr Bojanowski Matthijs Douze ... Ivan Laptev Natalia Neverova Gabriel Synnaeve Jakob Verbeek Hervé Jégou ViT 42 499 0 17 Jun 2021
Scene Transformer: A unified architecture for predicting multiple agent trajectories Jiquan Ngiam Benjamin Caine Vijay Vasudevan Zhengdong Zhang H. Chiang ... Ashish Venugopal David J. Weiss Benjamin Sapp Zhifeng Chen Jonathon Shlens 22 154 0 15 Jun 2021
Memory-efficient Transformers via Top- $k$ Attention Ankit Gupta Guy Dar Shaya Goodman David Ciprut Jonathan Berant MQ 40 51 0 13 Jun 2021
Going Beyond Linear Transformers with Recurrent Fast Weight Programmers Kazuki Irie Imanol Schlag Róbert Csordás Jürgen Schmidhuber 33 57 0 11 Jun 2021
Gaussian Bounding Boxes and Probabilistic Intersection-over-Union for Object Detection Jeffri Murrugarra-Llerena Luis Felipe de Araujo Zeni Lucas N. Kirsten C. R. Jung 31 33 0 10 Jun 2021
Transformed CNNs: recasting pre-trained convolutional layers with self-attention Stéphane dÁscoli Levent Sagun Giulio Biroli Ari S. Morcos ViT 13 6 0 10 Jun 2021
CoAtNet: Marrying Convolution and Attention for All Data Sizes Zihang Dai Hanxiao Liu Quoc V. Le Mingxing Tan ViT 49 1,167 0 09 Jun 2021
Refiner: Refining Self-attention for Vision Transformers Daquan Zhou Yujun Shi Bingyi Kang Weihao Yu Zihang Jiang Yuan Li Xiaojie Jin Qibin Hou Jiashi Feng ViT 29 59 0 07 Jun 2021
RegionViT: Regional-to-Local Attention for Vision Transformers Chun-Fu Chen Rameswar Panda Quanfu Fan ViT 16 194 0 04 Jun 2021
Attention mechanisms and deep learning for machine vision: A survey of the state of the art A. M. Hafiz S. A. Parah R. A. Bhat 21 45 0 03 Jun 2021
Analogous to Evolutionary Algorithm: Designing a Unified Sequence Model Jiangning Zhang Chao Xu Jian Li Wenzhou Chen Yabiao Wang Ying Tai Shuo Chen Chengjie Wang Feiyue Huang Yong Liu 32 22 0 31 May 2021
An Attention Free Transformer Shuangfei Zhai Walter A. Talbott Nitish Srivastava Chen Huang Hanlin Goh Ruixiang Zhang J. Susskind ViT 35 127 0 28 May 2021
KVT: k-NN Attention for Boosting Vision Transformers Pichao Wang Xue Wang F. Wang Ming Lin Shuning Chang Hao Li R. L. Jin ViT 51 105 0 28 May 2021
Nested Hierarchical Transformer: Towards Accurate, Data-Efficient and Interpretable Visual Understanding Zizhao Zhang Han Zhang Long Zhao Ting Chen Sercan Ö. Arik Tomas Pfister ViT 17 169 0 26 May 2021
Relative Positional Encoding for Transformers with Linear Complexity Antoine Liutkus Ondřej Cífka Shih-Lun Wu Umut Simsekli Yi-Hsuan Yang Gaël Richard 33 44 0 18 May 2021
Pay Attention to MLPs Hanxiao Liu Zihang Dai David R. So Quoc V. Le AI4CE 57 651 0 17 May 2021
T-EMDE: Sketching-based global similarity for cross-modal retrieval Barbara Rychalska Mikolaj Wieczorek Jacek Dąbrowski 33 0 0 10 May 2021
MLP-Mixer: An all-MLP Architecture for Vision Ilya O. Tolstikhin N. Houlsby Alexander Kolesnikov Lucas Beyer Xiaohua Zhai ... Andreas Steiner Daniel Keysers Jakob Uszkoreit Mario Lucic Alexey Dosovitskiy 274 2,603 0 04 May 2021
LambdaUNet: 2.5D Stroke Lesion Segmentation of Diffusion-weighted MR Images Yanglan Ou Ye Yuan Xiaolei Huang Kelvin K. Wong John Volpi J. Z. Wang Stephen T. C. Wong MedIm 11 8 0 28 Apr 2021
Vision Transformers with Patch Diversification Chengyue Gong Dilin Wang Meng Li Vikas Chandra Qiang Liu ViT 42 62 0 26 Apr 2021
Visformer: The Vision-friendly Transformer Zhengsu Chen Lingxi Xie Jianwei Niu Xuefeng Liu Longhui Wei Qi Tian ViT 120 209 0 26 Apr 2021
Multiscale Vision Transformers Haoqi Fan Bo Xiong K. Mangalam Yanghao Li Zhicheng Yan Jitendra Malik Christoph Feichtenhofer ViT 63 1,224 0 22 Apr 2021
SoT: Delving Deeper into Classification Head for Transformer Jiangtao Xie Rui Zeng Qilong Wang Ziqi Zhou P. Li ViT 31 12 0 22 Apr 2021
All Tokens Matter: Token Labeling for Training Better Vision Transformers Zihang Jiang Qibin Hou Li-xin Yuan Daquan Zhou Yujun Shi Xiaojie Jin Anran Wang Jiashi Feng ViT 19 203 0 22 Apr 2021
Efficient conformer-based speech recognition with linear attention Shengqiang Li Menglong Xu Xiao-Lei Zhang 21 20 0 14 Apr 2021
Co-Scale Conv-Attentional Image Transformers Weijian Xu Yifan Xu Tyler A. Chang Z. Tu ViT 24 373 0 13 Apr 2021
Fourier Image Transformer T. Buchholz Florian Jug ViT 17 17 0 06 Apr 2021
LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inference Ben Graham Alaaeldin El-Nouby Hugo Touvron Pierre Stock Armand Joulin Hervé Jégou Matthijs Douze ViT 22 768 0 02 Apr 2021
EfficientNetV2: Smaller Models and Faster Training Mingxing Tan Quoc V. Le EgoV 14 2,610 0 01 Apr 2021
Going deeper with Image Transformers Hugo Touvron Matthieu Cord Alexandre Sablayrolles Gabriel Synnaeve Hervé Jégou ViT 27 986 0 31 Mar 2021
Rethinking Spatial Dimensions of Vision Transformers Byeongho Heo Sangdoo Yun Dongyoon Han Sanghyuk Chun Junsuk Choe Seong Joon Oh ViT 362 564 0 30 Mar 2021
CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification Chun-Fu Chen Quanfu Fan Rameswar Panda ViT 28 1,424 0 27 Mar 2021
A Practical Survey on Faster and Lighter Transformers Quentin Fournier G. Caron Daniel Aloise 14 93 0 26 Mar 2021
Scaling Local Self-Attention for Parameter Efficient Visual Backbones Ashish Vaswani Prajit Ramachandran A. Srinivas Niki Parmar Blake A. Hechtman Jonathon Shlens 18 395 0 23 Mar 2021
BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search Changlin Li Tao Tang Guangrun Wang Jiefeng Peng Bing Wang Xiaodan Liang Xiaojun Chang ViT 46 105 0 23 Mar 2021
Incorporating Convolution Designs into Visual Transformers Kun Yuan Shaopeng Guo Ziwei Liu Aojun Zhou F. Yu Wei Wu ViT 56 467 0 22 Mar 2021
UNETR: Transformers for 3D Medical Image Segmentation Ali Hatamizadeh Yucheng Tang Vishwesh Nath Dong Yang Andriy Myronenko Bennett Landman H. Roth Daguang Xu ViT MedIm 92 1,535 0 18 Mar 2021
Value-aware Approximate Attention Ankit Gupta Jonathan Berant 13 5 0 17 Mar 2021
Revisiting ResNets: Improved Training and Scaling Strategies Irwan Bello W. Fedus Xianzhi Du E. D. Cubuk A. Srinivas Nayeon Lee Jonathon Shlens Barret Zoph 29 297 0 13 Mar 2021
Involution: Inverting the Inherence of Convolution for Visual Recognition Duo Li Jie Hu Changhu Wang Xiangtai Li Qi She Lei Zhu Tong Zhang Qifeng Chen BDL 19 304 0 10 Mar 2021
Perception Framework through Real-Time Semantic Segmentation and Scene Recognition on a Wearable System for the Visually Impaired Yingzhi Zhang Haoye Chen Kailun Yang Jiaming Zhang Rainer Stiefelhagen 18 6 0 06 Mar 2021
Perceiver: General Perception with Iterative Attention Andrew Jaegle Felix Gimeno Andrew Brock Andrew Zisserman Oriol Vinyals João Carreira VLM ViT MDE 85 973 0 04 Mar 2021
Task-Adaptive Neural Network Search with Meta-Contrastive Learning Wonyong Jeong Hayeon Lee G. Park Eunyoung Hyung Jinheon Baek Sung Ju Hwang 16 9 0 02 Mar 2021
Generative Adversarial Transformers Drew A. Hudson C. L. Zitnick ViT 25 179 0 01 Mar 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 289 3,623 0 24 Feb 2021