Associative Recurrent Memory Transformer

17 February 2025

Papers citing "Associative Recurrent Memory Transformer"

24 / 24 papers shown

Title
Mechanistic evaluation of Transformers and state space models Aryaman Arora Neil Rathi Nikil Roashan Selvam Róbert Csordás Dan Jurafsky Christopher Potts 68 0 0 21 May 2025
BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack Yuri Kuratov Aydar Bulatov Petr Anokhin Ivan Rodkin Dmitry Sorokin Artyom Sorokin Andrey Kravchenko RALM ALM LRM ReLM ELM 66 71 0 14 Jun 2024
In Search of Needles in a 11M Haystack: Recurrent Memory Finds What LLMs Miss Yuri Kuratov Aydar Bulatov Petr Anokhin Dmitry Sorokin Artyom Sorokin Andrey Kravchenko RALM 134 34 0 16 Feb 2024
Repeat After Me: Transformers are Better than State Space Models at Copying Samy Jelassi David Brandfonbrener Sham Kakade Eran Malach 118 88 0 01 Feb 2024
Efficient Streaming Language Models with Attention Sinks Michel Lang Yuandong Tian Beidi Chen Song Han Mike Lewis AI4TS RALM 107 705 0 29 Sep 2023
YaRN: Efficient Context Window Extension of Large Language Models Bowen Peng Jeffrey Quesnelle Honglu Fan Enrico Shippole OSLM 53 240 0 31 Aug 2023
Retentive Network: A Successor to Transformer for Large Language Models Yutao Sun Li Dong Shaohan Huang Shuming Ma Yuqing Xia Jilong Xue Jianyong Wang Furu Wei LRM 95 317 0 17 Jul 2023
Long-range Language Modeling with Self-retrieval Ohad Rubin Jonathan Berant RALM KELM 67 19 0 23 Jun 2023
RWKV: Reinventing RNNs for the Transformer Era Bo Peng Eric Alcaide Quentin G. Anthony Alon Albalak Samuel Arcadinho ... Qihang Zhao P. Zhou Qinghua Zhou Jian Zhu Rui-Jie Zhu 173 585 0 22 May 2023
LLaMA: Open and Efficient Foundation Language Models Hugo Touvron Thibaut Lavril Gautier Izacard Xavier Martinet Marie-Anne Lachaux ... Faisal Azhar Aurelien Rodriguez Armand Joulin Edouard Grave Guillaume Lample ALM PILM 964 12,840 0 27 Feb 2023
REPLUG: Retrieval-Augmented Black-Box Language Models Weijia Shi Sewon Min Michihiro Yasunaga Minjoon Seo Rich James M. Lewis Luke Zettlemoyer Wen-tau Yih RALM VLM KELM 118 611 0 30 Jan 2023
Hungry Hungry Hippos: Towards Language Modeling with State Space Models Daniel Y. Fu Tri Dao Khaled Kamal Saab A. Thomas Atri Rudra Christopher Ré 89 383 0 28 Dec 2022
Memorizing Transformers Yuhuai Wu M. Rabe DeLesley S. Hutchins Christian Szegedy RALM 64 175 0 16 Mar 2022
Improving language models by retrieving from trillions of tokens Sebastian Borgeaud A. Mensch Jordan Hoffmann Trevor Cai Eliza Rutherford ... Simon Osindero Karen Simonyan Jack W. Rae Erich Elsen Laurent Sifre KELM RALM 180 1,069 0 08 Dec 2021
Efficiently Modeling Long Sequences with Structured State Spaces Albert Gu Karan Goel Christopher Ré 180 1,719 0 31 Oct 2021
Linear Transformers Are Secretly Fast Weight Programmers Imanol Schlag Kazuki Irie Jürgen Schmidhuber 92 236 0 22 Feb 2021
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 404 2,051 0 31 Dec 2020
Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention Angelos Katharopoulos Apoorv Vyas Nikolaos Pappas Franccois Fleuret 153 1,734 0 29 Jun 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 546 41,106 0 28 May 2020
REALM: Retrieval-Augmented Language Model Pre-Training Kelvin Guu Kenton Lee Zora Tung Panupong Pasupat Ming-Wei Chang RALM 96 2,050 0 10 Feb 2020
Compressive Transformers for Long-Range Sequence Modelling Jack W. Rae Anna Potapenko Siddhant M. Jayakumar Timothy Lillicrap RALM VLM KELM 60 636 0 13 Nov 2019
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context Zihang Dai Zhilin Yang Yiming Yang J. Carbonell Quoc V. Le Ruslan Salakhutdinov VLM 163 3,714 0 09 Jan 2019
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 521 129,831 0 12 Jun 2017
Memory Networks Jason Weston S. Chopra Antoine Bordes GNN KELM 141 1,702 0 15 Oct 2014