RCMHA: Relative Convolutional Multi-Head Attention for Natural Language Modelling

7 August 2023

Papers citing "RCMHA: Relative Convolutional Multi-Head Attention for Natural Language Modelling"

17 / 17 papers shown

Title
Primer: Searching for Efficient Transformers for Language Modeling David R. So Wojciech Mañke Hanxiao Liu Zihang Dai Noam M. Shazeer Quoc V. Le VLM 248 156 0 17 Sep 2021
CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification Chun-Fu Chen Quanfu Fan Yikang Shen ViT 68 1,478 0 27 Mar 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 518 3,718 0 24 Feb 2021
Object-Centric Learning with Slot Attention Francesco Locatello Dirk Weissenborn Thomas Unterthiner Aravindh Mahendran G. Heigold Jakob Uszkoreit Alexey Dosovitskiy Thomas Kipf OCL 214 845 0 26 Jun 2020
Linformer: Self-Attention with Linear Complexity Sinong Wang Belinda Z. Li Madian Khabsa Han Fang Hao Ma 210 1,702 0 08 Jun 2020
Single Headed Attention RNN: Stop Thinking With Your Head Stephen Merity 55 68 0 26 Nov 2019
Generating Long Sequences with Sparse Transformers R. Child Scott Gray Alec Radford Ilya Sutskever 107 1,896 0 23 Apr 2019
Calibration of Encoder Decoder Models for Neural Machine Translation Aviral Kumar Sunita Sarawagi 166 100 0 03 Mar 2019
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context Zihang Dai Zhilin Yang Yiming Yang J. Carbonell Quoc V. Le Ruslan Salakhutdinov VLM 216 3,726 0 09 Jan 2019
CBAM: Convolutional Block Attention Module Sanghyun Woo Jongchan Park Joon-Young Lee In So Kweon 222 16,541 0 17 Jul 2018
Non-local Neural Networks Xinyu Wang Ross B. Girshick Abhinav Gupta Kaiming He OffRL 283 8,905 0 21 Nov 2017
Squeeze-and-Excitation Networks Jie Hu Li Shen Samuel Albanie Gang Sun Enhua Wu 424 26,481 0 05 Sep 2017
Structured Attention Networks Yoon Kim Carl Denton Luong Hoang Alexander M. Rush 106 463 0 03 Feb 2017
Neural Machine Translation with Supervised Attention Lemao Liu Masao Utiyama A. Finch Eiichiro Sumita 68 156 0 14 Sep 2016
Attention-over-Attention Neural Networks for Reading Comprehension Yiming Cui Zhongfu Chen Si Wei Shijin Wang Ting Liu Guoping Hu AIMat 66 439 0 15 Jul 2016
Neural Machine Translation by Jointly Learning to Align and Translate Dzmitry Bahdanau Kyunghyun Cho Yoshua Bengio AIMat 546 27,295 0 01 Sep 2014
Deep Networks with Internal Selective Attention through Feedback Connections Marijn F. Stollenga Jonathan Masci Faustino J. Gomez Jürgen Schmidhuber 160 257 0 11 Jul 2014