PermuteFormer: Efficient Relative Position Encoding for Long Sequences

6 September 2021

Papers citing "PermuteFormer: Efficient Relative Position Encoding for Long Sequences"

14 / 14 papers shown

Title
SMR: State Memory Replay for Long Sequence Modeling Biqing Qi Junqi Gao Kaiyan Zhang Dong Li Jianxing Liu Ligang Wu Bowen Zhou 33 5 0 27 May 2024
LLM Inference Unveiled: Survey and Roofline Model Insights Zhihang Yuan Yuzhang Shang Yang Zhou Zhen Dong Zhe Zhou ... Yong Jae Lee Yan Yan Beidi Chen Guangyu Sun Kurt Keutzer 45 79 0 26 Feb 2024
Advancing Transformer Architecture in Long-Context Large Language Models: A Comprehensive Survey Yunpeng Huang Jingwei Xu Junyu Lai Zixu Jiang Taolue Chen ... Xiaoxing Ma Lijuan Yang Zhou Xin Shupeng Li Penghao Zhao LLMAG KELM 36 54 0 21 Nov 2023
Transformers as Graph-to-Graph Models James Henderson Alireza Mohammadshahi Andrei Catalin Coman Lesly Miculicich GNN 27 6 0 27 Oct 2023
Linearized Relative Positional Encoding Zhen Qin Weixuan Sun Kaiyue Lu Huizhong Deng Dong Li Xiaodong Han Yuchao Dai Lingpeng Kong Yiran Zhong 20 12 0 18 Jul 2023
ContextSpeech: Expressive and Efficient Text-to-Speech for Paragraph Reading Yujia Xiao Shaofei Zhang Xi Wang Xuejiao Tan Lei He Sheng Zhao Frank Soong Tan Lee 25 5 0 03 Jul 2023
MH-DETR: Video Moment and Highlight Detection with Cross-modal Transformer Yifang Xu Yunzhuo Sun Yang Li Yilei Shi Xiaoxia Zhu S. Du ViT 51 33 0 29 Apr 2023
Efficient Attention via Control Variates Lin Zheng Jianbo Yuan Chong-Jun Wang Lingpeng Kong 34 18 0 09 Feb 2023
Lightweight Structure-Aware Attention for Visual Understanding Heeseung Kwon F. M. Castro M. Marín-Jiménez N. Guil Alahari Karteek 28 2 0 29 Nov 2022
Word Order Matters when you Increase Masking Karim Lasri Alessandro Lenci Thierry Poibeau 36 7 0 08 Nov 2022
Neural Architectures for Biological Inter-Sentence Relation Extraction Enrique Noriega-Atala Peter Lovett Clayton T. Morrison Mihai Surdeanu NAI 33 3 0 17 Dec 2021
On Learning the Transformer Kernel Sankalan Pal Chowdhury Adamos Solomou Kumar Avinava Dubey Mrinmaya Sachan ViT 52 14 0 15 Oct 2021
Ripple Attention for Visual Perception with Sub-quadratic Complexity Lin Zheng Huijie Pan Lingpeng Kong 26 3 0 06 Oct 2021
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 285 2,015 0 28 Jul 2020