Memformer: A Memory-Augmented Transformer for Sequence Modeling

Memformer: A Memory-Augmented Transformer for Sequence Modeling

14 October 2020

Kun Qian

Papers citing "Memformer: A Memory-Augmented Transformer for Sequence Modeling"

19 / 19 papers shown

Title
Sequential-NIAH: A Needle-In-A-Haystack Benchmark for Extracting Sequential Needles from Long Contexts Yifei Yu Qian Zhang Lingfeng Qiao Di Yin Fang Li Jie Wang Z. Chen Suncong Zheng Xiaolong Liang Xingchen Sun 44 0 0 07 Apr 2025
LCIRC: A Recurrent Compression Approach for Efficient Long-form Context and Query Dependent Modeling in LLMs Sumin An Junyoung Sung Wonpyo Park Chanjun Park Paul Hongsuck Seo 100 0 0 10 Feb 2025
Episodic memory in AI agents poses risks that should be studied and mitigated Chad DeChant 67 2 0 20 Jan 2025
Learning to Reason Iteratively and Parallelly for Complex Visual Reasoning Scenarios Shantanu Jaiswal Debaditya Roy Basura Fernando Cheston Tan ReLM LRM 79 2 0 20 Nov 2024
Token Turing Machines are Efficient Vision Models Purvish Jajal Nick Eliopoulos Benjamin Shiue-Hal Chou George K. Thiravathukal James C. Davis Yung-Hsiang Lu 98 0 0 11 Sep 2024
You Only Use Reactive Attention Slice For Long Context Retrieval Yun Joon Soh Hanxian Huang Yuandong Tian Jishen Zhao RALM 46 0 0 03 Sep 2024
BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack Yuri Kuratov Aydar Bulatov Petr Anokhin Ivan Rodkin Dmitry Sorokin Artyom Sorokin Andrey Kravchenko RALM ALM LRM ReLM ELM 51 61 0 14 Jun 2024
MemLLM: Finetuning LLMs to Use An Explicit Read-Write Memory Ali Modarressi Abdullatif Köksal Ayyoob Imani Mohsen Fayyaz Hinrich Schütze KELM 112 9 0 17 Apr 2024
NovelQA: Benchmarking Question Answering on Documents Exceeding 200K Tokens Cunxiang Wang Ruoxi Ning Boqi Pan Tonghui Wu Qipeng Guo ... Guangsheng Bao Xiangkun Hu Zheng Zhang Qian Wang Yue Zhang RALM 106 4 0 18 Mar 2024
Sound Source Separation Using Latent Variational Block-Wise Disentanglement Karim Helwani M. Togami Paris Smaragdis Michael M. Goodwin BDL DRL 26 1 0 08 Feb 2024
Investigating Recurrent Transformers with Dynamic Halt Jishnu Ray Chowdhury Cornelia Caragea 39 1 0 01 Feb 2024
Uncertainty Guided Global Memory Improves Multi-Hop Question Answering Alsu Sagirova Andrey Kravchenko RALM 28 1 0 29 Nov 2023
Recursively Summarizing Enables Long-Term Dialogue Memory in Large Language Models Qingyue Wang Y. Fu Yanan Cao Zhiliang Tian Shi Wang Dacheng Tao LLMAG KELM RALM 67 24 0 29 Aug 2023
A Memory-Augmented Multi-Task Collaborative Framework for Unsupervised Traffic Accident Detection in Driving Videos Rongqin Liang Yuanman Li Yingxin Yi Jiantao Zhou Xia Li 11 2 0 27 Jul 2023
Memory Efficient Neural Processes via Constant Memory Attention Block Leo Feng Frederick Tung Hossein Hajimirsadeghi Yoshua Bengio Mohamed Osama Ahmed 31 5 0 23 May 2023
Scaling Transformer to 1M tokens and beyond with RMT Aydar Bulatov Yuri Kuratov Yermek Kapushev Andrey Kravchenko LRM 25 87 0 19 Apr 2023
Improving Autoregressive NLP Tasks via Modular Linearized Attention Victor Agostinelli Lizhong Chen 27 1 0 17 Apr 2023
Zero-Shot Text-to-Image Generation Aditya A. Ramesh Mikhail Pavlov Gabriel Goh Scott Gray Chelsea Voss Alec Radford Mark Chen Ilya Sutskever VLM 255 4,796 0 24 Feb 2021
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 285 2,017 0 28 Jul 2020