SHAPE: Shifted Absolute Position Embedding for Transformers

13 September 2021

Papers citing "SHAPE: Shifted Absolute Position Embedding for Transformers"

9 / 9 papers shown

Title
Investigating Length Issues in Document-level Machine Translation Ziqian Peng Rachel Bawden François Yvon 69 1 0 23 Dec 2024
Positional Encoding Helps Recurrent Neural Networks Handle a Large Vocabulary Takashi Morita 16 3 0 31 Jan 2024
Positional Description Matters for Transformers Arithmetic Ruoqi Shen Sébastien Bubeck Ronen Eldan Yin Tat Lee Yuanzhi Li Yi Zhang 34 37 0 22 Nov 2023
Long-MIL: Scaling Long Contextual Multiple Instance Learning for Histopathology Whole Slide Image Analysis Honglin Li Yunlong Zhang Chenglu Zhu Jiatong Cai Sunyi Zheng Lin Yang VLM 35 4 0 21 Nov 2023
The Curious Case of Absolute Position Embeddings Koustuv Sinha Amirhossein Kazemnejad Siva Reddy J. Pineau Dieuwke Hupkes Adina Williams 87 15 0 23 Oct 2022
KERPLE: Kernelized Relative Positional Embedding for Length Extrapolation Ta-Chung Chi Ting-Han Fan Peter J. Ramadge Alexander I. Rudnicky 41 65 0 20 May 2022
Continual Transformers: Redundancy-Free Attention for Online Inference Lukas Hedegaard Arian Bakhtiarnia Alexandros Iosifidis CLL 27 11 0 17 Jan 2022
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 282 2,015 0 28 Jul 2020
OpenNMT: Open-Source Toolkit for Neural Machine Translation Guillaume Klein Yoon Kim Yuntian Deng Jean Senellart Alexander M. Rush 262 1,896 0 10 Jan 2017