Title
ExpertRAG: Efficient RAG with Mixture of Experts -- Optimizing Context Retrieval for Adaptive LLM Responses Esmail Gumaan MoE 30 0 0 23 Mar 2025
Sparse Brains are Also Adaptive Brains: Cognitive-Load-Aware Dynamic Activation for LLMs Yiheng Yang Yujie Wang Chi Ma Lei Yu Emmanuele Chersoni Chu-Ren Huang 79 0 0 26 Feb 2025
Rethinking Addressing in Language Models via Contexualized Equivariant Positional Encoding Jiajun Zhu Peihao Wang Ruisi Cai Jason D. Lee Pan Li Zhilin Wang KELM 48 1 0 03 Jan 2025
MDCure: A Scalable Pipeline for Multi-Document Instruction-Following Gabrielle Kaili-May Liu Bowen Shi Avi Caciularu Idan Szpektor Arman Cohan 72 4 0 30 Oct 2024
Keep Guessing? When Considering Inference Scaling, Mind the Baselines G. Yona Or Honovich Omer Levy Roee Aharoni UQLM LRM 33 0 0 20 Oct 2024
Assessing Episodic Memory in LLMs with Sequence Order Recall Tasks Mathis Pink Vy A. Vo Qinyuan Wu Jianing Mu Javier S. Turek Uri Hasson K. A. Norman Sebastian Michelmann Alexander G. Huth Mariya Toneva 26 1 0 10 Oct 2024
SEGMENT+: Long Text Processing with Short-Context Language Models Wei Shi Shuang Li Kerun Yu Jinglei Chen Zujie Liang ... Feng Wei Bo Zheng Jiaqing Liang Jiangjie Chen Yanghua Xiao RALM VLM 57 2 0 09 Oct 2024
HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughly Howard Yen Tianyu Gao Minmin Hou Ke Ding Daniel Fleischer Peter Izsak Moshe Wasserblat Danqi Chen ALM ELM 62 25 0 03 Oct 2024
First Activations Matter: Training-Free Methods for Dynamic Activation in Large Language Models Chi Ma Mincong Huang Ying Zhang Chao Wang Yujie Wang Lei Yu Chuan Liu Wei Lin AI4CE LLMSV 51 2 0 21 Aug 2024
MovieSum: An Abstractive Summarization Dataset for Movie Screenplays Rohit Saxena Frank Keller 21 3 0 12 Aug 2024
Stress-Testing Long-Context Language Models with Lifelong ICL and Task Haystack Xiaoyue Xu Qinyuan Ye Xiang Ren 47 6 0 23 Jul 2024
ChatQA 2: Bridging the Gap to Proprietary LLMs in Long Context and RAG Capabilities Peng-Tao Xu Ming-Yu Liu Xianchao Wu Zihan Liu M. Shoeybi Mohammad Shoeybi Bryan Catanzaro RALM 52 14 0 19 Jul 2024
AdaptEval: Evaluating Large Language Models on Domain Adaptation for Text Summarization Anum Afzal Ribin Chalumattu Florian Matthes Laura Mascarell ALM ELM 27 4 0 16 Jul 2024
FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision Jay Shah Ganesh Bikshandi Ying Zhang Vijay Thakkar Pradeep Ramani Tri Dao 50 113 0 11 Jul 2024
HDT: Hierarchical Document Transformer Haoyu He Markus Flicke Jan Buchmann Iryna Gurevych Andreas Geiger 43 0 0 11 Jul 2024
How Well Can a Long Sequence Model Model Long Sequences? Comparing Architechtural Inductive Biases on Long-Context Abilities Jerry Huang 57 7 0 11 Jul 2024
B'MOJO: Hybrid State Space Realizations of Foundation Models with Eidetic and Fading Memory L. Zancato Arjun Seshadri Yonatan Dukler Aditya Golatkar Yantao Shen Benjamin Bowman Matthew Trager Alessandro Achille Stefano Soatto 34 8 0 08 Jul 2024
MMLongBench-Doc: Benchmarking Long-context Document Understanding with Visualizations Yubo Ma Yuhang Zang Liangyu Chen Meiqi Chen Yizhu Jiao ... Liangming Pan Yu-Gang Jiang Jiaqi Wang Yixin Cao Aixin Sun ELM RALM VLM 31 24 0 01 Jul 2024
Is It Really Long Context if All You Need Is Retrieval? Towards Genuinely Difficult Long Context NLP Omer Goldman Alon Jacovi Aviv Slobodkin Aviya Maimon Ido Dagan Reut Tsarfaty 64 11 0 29 Jun 2024
Suri: Multi-constraint Instruction Following for Long-form Text Generation Chau Minh Pham Simeng Sun Mohit Iyyer ALM LRM 45 15 0 27 Jun 2024
CLERC: A Dataset for Legal Case Retrieval and Retrieval-Augmented Analysis Generation Abe Bohan Hou Orion Weller Guanghui Qin Eugene Yang Dawn J Lawrie Nils Holzenberger Andrew Blair-Stanek Benjamin Van Durme AILaw ELM 87 5 0 24 Jun 2024
SEAM: A Stochastic Benchmark for Multi-Document Tasks Gili Lior Avi Caciularu Arie Cattan Shahar Levy Ori Shapira Gabriel Stanovsky RALM 40 4 0 23 Jun 2024
Insights into LLM Long-Context Failures: When Transformers Know but Don't Tell Taiming Lu Muhan Gao Kuai Yu Adam Byerly Daniel Khashabi 49 12 0 20 Jun 2024
Can Few-shot Work in Long-Context? Recycling the Context to Generate Demonstrations Arie Cattan Alon Jacovi Alex Fabrikant Jonathan Herzig Roee Aharoni ... Dror Marcus Avinatan Hassidim Yossi Matias Idan Szpektor Avi Caciularu RALM 34 0 0 19 Jun 2024
Generating Tables from the Parametric Knowledge of Language Models Yevgeni Berkovitch Oren Glickman Amit Somech Tomer Wolfson LMTD 46 1 0 16 Jun 2024
BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack Yuri Kuratov Aydar Bulatov Petr Anokhin Ivan Rodkin Dmitry Sorokin Artyom Sorokin Mikhail Burtsev RALM ALM LRM ReLM ELM 49 59 0 14 Jun 2024
An Empirical Study of Mamba-based Language Models R. Waleffe Wonmin Byeon Duncan Riach Brandon Norick V. Korthikanti ... Vartika Singh Jared Casper Jan Kautz M. Shoeybi Bryan Catanzaro 61 65 0 12 Jun 2024
LoCoCo: Dropping In Convolutions for Long Context Compression Ruisi Cai Yuandong Tian Zhangyang Wang Beidi Chen 41 9 0 08 Jun 2024
Chain of Agents: Large Language Models Collaborating on Long-Context Tasks Yusen Zhang Ruoxi Sun Yanfei Chen Tomas Pfister Rui Zhang Sercan Ö. Arik RALM AI4CE LLMAG 54 28 0 04 Jun 2024
Various Lengths, Constant Speed: Efficient Language Modeling with Lightning Attention Zhen Qin Weigao Sun Dong Li Xuyang Shen Weixuan Sun Yiran Zhong 46 9 0 27 May 2024
Linearizing Large Language Models Jean-Pierre Mercat Igor Vasiljevic Sedrick Scott Keh Kushal Arora Achal Dave Adrien Gaidon Thomas Kollar 38 19 0 10 May 2024
LongEmbed: Extending Embedding Models for Long Context Retrieval Dawei Zhu Liang Wang Nan Yang Yifan Song Wenhao Wu Furu Wei Sujian Li RALM 43 21 0 18 Apr 2024
TransformerFAM: Feedback attention is working memory Dongseong Hwang Weiran Wang Zhuoyuan Huo K. Sim P. M. Mengibar 32 12 0 14 Apr 2024
Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length Xuezhe Ma Xiaomeng Yang Wenhan Xiong Beidi Chen Lili Yu Hao Zhang Jonathan May Luke Zettlemoyer Omer Levy Chunting Zhou 53 27 0 12 Apr 2024
HGRN2: Gated Linear RNNs with State Expansion Zhen Qin Songlin Yang Weixuan Sun Xuyang Shen Dong Li Weigao Sun Yiran Zhong LRM 47 47 0 11 Apr 2024
Select and Summarize: Scene Saliency for Movie Script Summarization Rohit Saxena Frank Keller 27 2 0 04 Apr 2024
Long-Context Language Modeling with Parallel Context Encoding Howard Yen Tianyu Gao Danqi Chen 33 43 0 26 Feb 2024
FanOutQA: A Multi-Hop, Multi-Document Question Answering Benchmark for Large Language Models Andrew Zhu Alyssa Hwang Liam Dugan Chris Callison-Burch ELM 50 0 0 21 Feb 2024
A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts Kuang-Huei Lee Xinyun Chen Hiroki Furuta John F. Canny Ian S. Fischer RALM 55 29 0 15 Feb 2024
Lissard: Long and Simple Sequential Reasoning Datasets M. Bueno R. Lotufo Rodrigo Nogueira RALM LRM 25 2 0 12 Feb 2024
Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuning Hao Zhao Maksym Andriushchenko Francesco Croce Nicolas Flammarion ALM 94 44 0 07 Feb 2024
Nomic Embed: Training a Reproducible Long Context Text Embedder Zach Nussbaum John X. Morris Brandon Duderstadt Andriy Mulyar 24 95 0 02 Feb 2024
LOCOST: State-Space Models for Long Document Abstractive Summarization Florian Le Bronnec Song Duong Mathieu Ravaut Alexandre Allauzen Nancy F. Chen Vincent Guigue Alberto Lumbreras Laure Soulier Patrick Gallinari 45 8 0 31 Jan 2024
Document Structure in Long Document Transformers Jan Buchmann Max Eichler Jan-Micha Bodensohn Ilia Kuznetsov Iryna Gurevych 13 2 0 31 Jan 2024
Two Stones Hit One Bird: Bilevel Positional Encoding for Better Length Extrapolation Zhenyu He Guhao Feng Shengjie Luo Kai-Bo Yang Liwei Wang Jingjing Xu Zhi Zhang Hongxia Yang Di He 29 14 0 29 Jan 2024
PROXYQA: An Alternative Framework for Evaluating Long-Form Text Generation with Large Language Models Haochen Tan Zhijiang Guo Zhan Shi Lu Xu Zhili Liu ... Xiaoguang Li Yasheng Wang Lifeng Shang Qun Liu Linqi Song 35 12 0 26 Jan 2024
Structured Packing in LLM Training Improves Long Context Utilization Konrad Staniszewski Szymon Tworkowski Sebastian Jaszczur Yu Zhao Henryk Michalewski Lukasz Kuciñski Piotr Milo's 41 13 0 28 Dec 2023
Multimodal Gen-AI for Fundamental Investment Research Lezhi Li Ting-Yu Chang Hai Wang 13 6 0 24 Dec 2023
Extending Context Window of Large Language Models via Semantic Compression Weizhi Fei Xueyan Niu Pingyi Zhou Lu Hou Bo Bai Lei Deng Wei Han 41 26 0 15 Dec 2023
Marathon: A Race Through the Realm of Long Context with Large Language Models Lei Zhang Yunshui Li Ziqiang Liu Jiaxi Yang Junhao Liu Longze Chen Run Luo Min Yang OffRL LRM 45 5 0 15 Dec 2023