On the Relationship between Self-Attention and Convolutional Layers

8 November 2019

Papers citing "On the Relationship between Self-Attention and Convolutional Layers"

19 / 269 papers shown

Title
MHSA-Net: Multi-Head Self-Attention Network for Occluded Person Re-Identification Hongchen Tan Xiuping Liu Baocai Yin Xin Li 20 78 0 10 Aug 2020
flexgrid2vec: Learning Efficient Visual Representations Vectors Ali Hamdi D. Kim Flora D. Salim SSL GNN 30 7 0 30 Jul 2020
Towards Learning Convolutions from Scratch Behnam Neyshabur SSL 220 71 0 27 Jul 2020
Data Movement Is All You Need: A Case Study on Optimizing Transformers A. Ivanov Nikoli Dryden Tal Ben-Nun Shigang Li Torsten Hoefler 36 131 0 30 Jun 2020
Multi-Head Attention: Collaborate Instead of Concatenate Jean-Baptiste Cordonnier Andreas Loukas Martin Jaggi 6 107 0 29 Jun 2020
Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention Angelos Katharopoulos Apoorv Vyas Nikolaos Pappas Franccois Fleuret 65 1,671 0 29 Jun 2020
ULSAM: Ultra-Lightweight Subspace Attention Module for Compact Convolutional Neural Networks Rajat Saini N. Jha B. K. Das Sparsh Mittal C.Krishna Mohan 29 83 0 26 Jun 2020
The Depth-to-Width Interplay in Self-Attention Yoav Levine Noam Wies Or Sharir Hofit Bata Amnon Shashua 30 45 0 22 Jun 2020
A Trainable Optimal Transport Embedding for Feature Aggregation and its Relationship to Attention Grégoire Mialon Dexiong Chen Alexandre d’Aspremont Julien Mairal OT 12 0 0 22 Jun 2020
Infinite attention: NNGP and NTK for deep attention networks Jiri Hron Yasaman Bahri Jascha Narain Sohl-Dickstein Roman Novak 8 112 0 18 Jun 2020
Neural Anisotropy Directions Guillermo Ortiz-Jiménez Apostolos Modas Seyed-Mohsen Moosavi-Dezfooli P. Frossard 34 16 0 17 Jun 2020
Sparse and Continuous Attention Mechanisms André F. T. Martins António Farinhas Marcos Vinícius Treviso Vlad Niculae P. Aguiar Mário A. T. Figueiredo 6 40 0 12 Jun 2020
Visual Transformers: Token-based Image Representation and Processing for Computer Vision Bichen Wu Chenfeng Xu Xiaoliang Dai Alvin Wan Peizhao Zhang Zhicheng Yan Masayoshi Tomizuka Joseph E. Gonzalez Kurt Keutzer Peter Vajda ViT 39 546 0 05 Jun 2020
General-Purpose User Embeddings based on Mobile App Usage Junqi Zhang Bing Bai Ye Lin Jian Liang Kun Bai Fei Wang 35 35 0 27 May 2020
End-to-End Object Detection with Transformers Nicolas Carion Francisco Massa Gabriel Synnaeve Nicolas Usunier Alexander Kirillov Sergey Zagoruyko ViT 3DV PINN 125 12,691 0 26 May 2020
Synthesizer: Rethinking Self-Attention in Transformer Models Yi Tay Dara Bahri Donald Metzler Da-Cheng Juan Zhe Zhao Che Zheng 17 332 0 02 May 2020
Neuroevolution of Self-Interpretable Agents Yujin Tang Duong Nguyen David R Ha 21 111 0 18 Mar 2020
ExGate: Externally Controlled Gating for Feature-based Attention in Artificial Neural Networks M.G.H. Cox A. Mishra 27 1 0 08 Nov 2018
Convolutional Neural Networks for Sentence Classification Yoon Kim AILaw VLM 258 13,368 0 25 Aug 2014