Improve Transformer Models with Better Relative Position Embeddings

Improve Transformer Models with Better Relative Position Embeddings

28 September 2020

Papers citing "Improve Transformer Models with Better Relative Position Embeddings"

11 / 61 papers shown

Title
BERT-like Pre-training for Symbolic Piano Music Classification Tasks Yi-Hui Chou I-Chun Chen Chin-Jui Chang Joann Ching Yi-Hsuan Yang 40 25 0 12 Jul 2021
Convolutions and Self-Attention: Re-interpreting Relative Positions in Pre-trained Language Models Tyler A. Chang Yifan Xu Weijian Xu Zhuowen Tu ViT 29 15 0 10 Jun 2021
CAPE: Encoding Relative Positions with Continuous Augmented Positional Embeddings Tatiana Likhomanenko Qiantong Xu Gabriel Synnaeve R. Collobert A. Rogozhnikov OOD ViT 33 55 0 06 Jun 2021
Relative Positional Encoding for Transformers with Linear Complexity Antoine Liutkus Ondřej Cífka Shih-Lun Wu Umut Simsekli Yi-Hsuan Yang Gaël Richard 38 45 0 18 May 2021
RoFormer: Enhanced Transformer with Rotary Position Embedding Jianlin Su Yu Lu Shengfeng Pan Ahmed Murtadha Bo Wen Yunfeng Liu 46 2,190 0 20 Apr 2021
Syntax-Aware Graph-to-Graph Transformer for Semantic Role Labelling Alireza Mohammadshahi James Henderson 31 12 0 15 Apr 2021
Symbolic integration by integrating learning models with different strengths and weaknesses Hazumi Kubota Y. Tokuoka Takahiro G. Yamada Akira Funahashi AIMat 28 4 0 09 Mar 2021
Investigating the Limitations of Transformers with Simple Arithmetic Tasks Rodrigo Nogueira Zhiying Jiang Jimmy J. Li LRM 24 123 0 25 Feb 2021
Position Information in Transformers: An Overview Philipp Dufter Martin Schmitt Hinrich Schütze 13 139 0 22 Feb 2021
FLERT: Document-Level Features for Named Entity Recognition Stefan Schweter Alan Akbik 22 111 0 13 Nov 2020
A Decomposable Attention Model for Natural Language Inference Ankur P. Parikh Oscar Täckström Dipanjan Das Jakob Uszkoreit 213 1,367 0 06 Jun 2016