LIVEJoin the current RTAI Connect sessionJoin now

LaMemo: Language Modeling with Look-Ahead Memory

LaMemo: Language Modeling with Look-Ahead Memory

15 April 2022

Rongsheng Zhang

Papers citing "LaMemo: Language Modeling with Look-Ahead Memory"

17 / 17 papers shown

Title
Do Long-Range Language Models Actually Use Long-Range Context? Simeng Sun Kalpesh Krishna Andrew Mattarella-Micke Mohit Iyyer RALM 59 84 0 19 Sep 2021
$$\infty$-former: Infinite Memory Transformer$ $\infty$ -former: Infinite Memory Transformer Pedro Henrique Martins Zita Marinho André F. T. Martins 79 11 0 01 Sep 2021
Long Range Arena: A Benchmark for Efficient Transformers Yi Tay Mostafa Dehghani Samira Abnar Songlin Yang Dara Bahri Philip Pham J. Rao Liu Yang Sebastian Ruder Donald Metzler 147 720 0 08 Nov 2020
Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention Angelos Katharopoulos Apoorv Vyas Nikolaos Pappas Franccois Fleuret 201 1,771 0 29 Jun 2020
Linformer: Self-Attention with Linear Complexity Sinong Wang Belinda Z. Li Madian Khabsa Han Fang Hao Ma 213 1,706 0 08 Jun 2020
Longformer: The Long-Document Transformer Iz Beltagy Matthew E. Peters Arman Cohan RALM VLM 171 4,071 0 10 Apr 2020
Adaptively Sparse Transformers Gonçalo M. Correia Vlad Niculae André F. T. Martins 84 256 0 30 Aug 2019
Adaptive Attention Span in Transformers Sainbayar Sukhbaatar Edouard Grave Piotr Bojanowski Armand Joulin 76 285 0 19 May 2019
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context Zihang Dai Zhilin Yang Yiming Yang J. Carbonell Quoc V. Le Ruslan Salakhutdinov VLM 241 3,730 0 09 Jan 2019
Long-Term Feature Banks for Detailed Video Understanding Chao-Yuan Wu Christoph Feichtenhofer Haoqi Fan Kaiming He Philipp Krahenbuhl Ross B. Girshick 171 480 0 12 Dec 2018
Adaptive Input Representations for Neural Language Modeling Alexei Baevski Michael Auli 101 390 0 28 Sep 2018
Sharp Nearby, Fuzzy Far Away: How Neural Language Models Use Context Urvashi Khandelwal He He Peng Qi Dan Jurafsky RALM 52 295 0 12 May 2018
Pointer Sentinel Mixture Models Stephen Merity Caiming Xiong James Bradbury R. Socher RALM 319 2,876 0 26 Sep 2016
Layer Normalization Jimmy Lei Ba J. Kiros Geoffrey E. Hinton 413 10,494 0 21 Jul 2016
Neural Turing Machines Alex Graves Greg Wayne Ivo Danihelka 97 2,328 0 20 Oct 2014
Memory Networks Jason Weston S. Chopra Antoine Bordes GNN KELM 147 1,706 0 15 Oct 2014
Neural Machine Translation by Jointly Learning to Align and Translate Dzmitry Bahdanau Kyunghyun Cho Yoshua Bengio AIMat 558 27,311 0 01 Sep 2014

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.