R-Transformer: Recurrent Neural Network Enhanced Transformer

R-Transformer: Recurrent Neural Network Enhanced Transformer

12 July 2019

Zitao Liu

Papers citing "R-Transformer: Recurrent Neural Network Enhanced Transformer"

18 / 18 papers shown

Title
Compact Recurrent Transformer with Persistent Memory Edison Mucllari Z. Daniels David C. Zhang Qiang Ye CLL VLM 51 0 0 02 May 2025
Sparse Low-Ranked Self-Attention Transformer for Remaining Useful Lifetime Prediction of Optical Fiber Amplifiers Dominic Schneider Lutz Rapp 32 0 0 22 Sep 2024
Ex3: Automatic Novel Writing by Extracting, Excelsior and Expanding Lei Huang Jiaming Guo Guanhua He Xishan Zhang Rui Zhang Shaohui Peng Shaoli Liu Tianshi Chen 31 2 0 16 Aug 2024
Efficient User Sequence Learning for Online Services via Compressed Graph Neural Networks Yucheng Wu Liyue Chen Yu Cheng Shuai Chen Jinyu Xu Leye Wang GNN 32 0 0 05 Jun 2024
Looped Transformers are Better at Learning Learning Algorithms Liu Yang Kangwook Lee Robert D. Nowak Dimitris Papailiopoulos 24 24 0 21 Nov 2023
Deep representation learning: Fundamentals, Perspectives, Applications, and Open Challenges K. T. Baghaei Amirreza Payandeh Pooya Fayyazsanavi Shahram Rahimi Zhiqian Chen Somayeh Bakhtiari Ramezani FaML AI4TS 35 6 0 27 Nov 2022
TASKED: Transformer-based Adversarial learning for human activity recognition using wearable sensors via Self-KnowledgE Distillation Sungho Suh Vitor Fortes Rey P. Lukowicz 30 55 0 14 Sep 2022
Modeling Long-term Dependencies and Short-term Correlations in Patient Journey Data with Temporal Attention Networks for Health Prediction Yuxi Liu Zhenhao Zhang Antonio Jimeno Yepes Flora D. Salim 18 10 0 13 Jul 2022
Exploring the sequence length bottleneck in the Transformer for Image Captioning Jiapeng Hu Roberto Cavicchioli Alessandro Capotondi ViT 38 3 0 07 Jul 2022
Recurrent Video Restoration Transformer with Guided Deformable Attention Christos Sakaridis Yuchen Fan Xiaoyu Xiang Rakesh Ranjan Eddy Ilg Simon Green Jingyun Liang Kaicheng Zhang Radu Timofte Luc Van Gool 42 152 0 05 Jun 2022
Research on Domain Information Mining and Theme Evolution of Scientific Papers Changwei Zheng Zhe Xue M. Liang Feifei Kou Zeli Guan AI4CE 27 0 0 18 Apr 2022
Block-Recurrent Transformers DeLesley S. Hutchins Imanol Schlag Yuhuai Wu Ethan Dyer Behnam Neyshabur 23 94 0 11 Mar 2022
Low-Rank Constraints for Fast Inference in Structured Models Justin T. Chiu Yuntian Deng Alexander M. Rush BDL 32 13 0 08 Jan 2022
A Survey of Transformers Tianyang Lin Yuxin Wang Xiangyang Liu Xipeng Qiu ViT 53 1,088 0 08 Jun 2021
Rethinking Neural Operations for Diverse Tasks Nicholas Roberts M. Khodak Tri Dao Liam Li Christopher Ré Ameet Talwalkar AI4CE 36 22 0 29 Mar 2021
TransMask: A Compact and Fast Speech Separation Model Based on Transformer Zining Zhang Bingsheng He Zhenjie Zhang 21 21 0 19 Feb 2021
Transformer Language Models with LSTM-based Cross-utterance Information Representation G. Sun C. Zhang P. Woodland 76 32 0 12 Feb 2021
Highway Transformer: Self-Gating Enhanced Self-Attentive Networks Yekun Chai Jin Shuo Xinwen Hou 23 16 0 17 Apr 2020