Unlimiformer: Long-Range Transformers with Unlimited Length Input

2 May 2023

Graham Neubig

Papers citing "Unlimiformer: Long-Range Transformers with Unlimited Length Input"

28 / 28 papers shown

Title
TextTIGER: Text-based Intelligent Generation with Entity Prompt Refinement for Text-to-Image Generation Shintaro Ozaki Kazuki Hayashi Yusuke Sakai Jingun Kwon Hidetaka Kamigaito Katsuhiko Hayashi Manabu Okumura Taro Watanabe VLM 88 0 0 25 Apr 2025
Cognitive Memory in Large Language Models Lianlei Shan Shixian Luo Zezhou Zhu Yu Yuan Yong Wu LLMAG KELM 160 1 0 03 Apr 2025
LV-XAttn: Distributed Cross-Attention for Long Visual Inputs in Multimodal Large Language Models Tzu-Tao Chang Shivaram Venkataraman VLM 173 0 0 04 Feb 2025
Context-Aware Hierarchical Merging for Long Document Summarization Litu Ou Mirella Lapata MoMe 198 1 0 03 Feb 2025
ZETA: Leveraging Z-order Curves for Efficient Top-k Attention Qiuhao Zeng Jerry Huang Peng Lu Gezheng Xu Boxing Chen Charles Ling Boyu Wang 49 1 0 24 Jan 2025
RetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrieval Di Liu Meng Chen Baotong Lu Huiqiang Jiang Zhenhua Han ... Kaipeng Zhang Cheng Chen Fan Yang Yuqing Yang Lili Qiu 52 29 0 03 Jan 2025
How to Train Long-Context Language Models (Effectively) Tianyu Gao Alexander Wettig Howard Yen Danqi Chen RALM 72 38 0 03 Oct 2024
HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughly Howard Yen Tianyu Gao Minmin Hou Ke Ding Daniel Fleischer Peter Izsak Moshe Wasserblat Danqi Chen ALM ELM 62 25 0 03 Oct 2024
You Only Use Reactive Attention Slice For Long Context Retrieval Yun Joon Soh Hanxian Huang Yuandong Tian Jishen Zhao RALM 46 0 0 03 Sep 2024
Ex3: Automatic Novel Writing by Extracting, Excelsior and Expanding Lei Huang Jiaming Guo Guanhua He Xishan Zhang Rui Zhang Shaohui Peng Shaoli Liu Tianshi Chen 28 2 0 16 Aug 2024
Mitigate Position Bias in Large Language Models via Scaling a Single Dimension Yijiong Yu Huiqiang Jiang Xufang Luo Qianhui Wu Chin-Yew Lin Dongsheng Li Yuqing Yang Yongfeng Huang L. Qiu 46 9 0 04 Jun 2024
In-Context Learning with Long-Context Models: An In-Depth Exploration Amanda Bertsch Maor Ivgi Uri Alon Jonathan Berant Matthew R. Gormley Matthew R. Gormley Graham Neubig ReLM AIMat 93 64 0 30 Apr 2024
A Modular Approach for Multimodal Summarization of TV Shows Louis Mahon Mirella Lapata 26 9 0 06 Mar 2024
MEMORY-VQ: Compression for Tractable Internet-Scale Memory Yury Zemlyanskiy Michiel de Jong Luke Vilnis Santiago Ontañón William W. Cohen Sumit Sanghai Joshua Ainslie RALM MQ 35 0 0 28 Aug 2023
Text2KGBench: A Benchmark for Ontology-Driven Knowledge Graph Generation from Text Nandana Mihindukulasooriya Sanju Tiwari Carlos F. Enguix K. Lata 31 52 0 04 Aug 2023
Tool Documentation Enables Zero-Shot Tool-Usage with Large Language Models Cheng-Yu Hsieh Sibei Chen Chun-Liang Li Yasuhisa Fujii Alexander Ratner Chen-Yu Lee Ranjay Krishna Tomas Pfister LLMAG SyDa 43 41 0 01 Aug 2023
Multilingual Code Co-Evolution Using Large Language Models Jiyang Zhang Pengyu Nie Junyi Jessy Li Miloš Gligorić 32 20 0 27 Jul 2023
Long-range Language Modeling with Self-retrieval Ohad Rubin Jonathan Berant RALM KELM 19 18 0 23 Jun 2023
ZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understanding Uri Shaham Maor Ivgi Avia Efrat Jonathan Berant Omer Levy VLM 35 126 0 23 May 2023
You can't pick your neighbors, or can you? When and how to rely on retrieval in the $k$ NN-LM Andrew Drozdov Shufan Wang Razieh Rahimi Andrew McCallum Hamed Zamani Mohit Iyyer RALM 119 17 0 28 Oct 2022
An Empirical Survey on Long Document Summarization: Datasets, Models and Metrics Huan Yee Koh Jiaxin Ju Ming Liu Shirui Pan 78 122 0 03 Jul 2022
Training Language Models with Memory Augmentation Zexuan Zhong Tao Lei Danqi Chen RALM 236 128 0 25 May 2022
Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval Uri Alon Frank F. Xu Junxian He Sudipta Sengupta Dan Roth Graham Neubig RALM 77 62 0 28 Jan 2022
PRIMERA: Pyramid-based Masked Sentence Pre-training for Multi-document Summarization Wen Xiao Iz Beltagy Giuseppe Carenini Arman Cohan CVBM 83 114 0 16 Oct 2021
ContractNLI: A Dataset for Document-level Natural Language Inference for Contracts Yuta Koreeda Christopher D. Manning AILaw 94 96 0 05 Oct 2021
Leveraging Pretrained Models for Automatic Summarization of Doctor-Patient Conversations Longxiang Zhang Renato M. P. Negrinho Arindam Ghosh V. Jagannathan H. Hassanzadeh Thomas Schaaf Matthew R. Gormley LM&MA AI4MH 82 67 0 24 Sep 2021
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 282 2,015 0 28 Jul 2020
Efficient Content-Based Sparse Attention with Routing Transformers Aurko Roy M. Saffar Ashish Vaswani David Grangier MoE 246 580 0 12 Mar 2020