L-Eval: Instituting Standardized Evaluation for Long Context Language Models

20 July 2023

Lingpeng Kong

Xipeng Qiu

ELM

ALM

ArXiv PDF HTML

Papers citing "L-Eval: Instituting Standardized Evaluation for Long Context Language Models"

37 / 37 papers shown

Title
PSC: Extending Context Window of Large Language Models via Phase Shift Calibration Wenqiao Zhu Chao Xu Lulu Wang Jun Wu 15 1 0 18 May 2025
SAS-Bench: A Fine-Grained Benchmark for Evaluating Short Answer Scoring with Large Language Models Peichao Lai Kaipeng Zhang Yi Lin L. Zhang Feiyang Ye ... Zifei Shan Zeang Sheng Yansen Wang Wentao Zhang Bin Cui ELM LRM 49 0 0 12 May 2025
Rethinking Memory in AI: Taxonomy, Operations, Topics, and Future Directions Yiming Du Wenyu Huang Danna Zheng Zhaowei Wang Sébastien Montella Mirella Lapata Kam-Fai Wong Jeff Z. Pan KELM MU 86 2 0 01 May 2025
Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarks Yixin Cao Shibo Hong Xuzhao Li Jiahao Ying Yubo Ma ... Juanzi Li Aixin Sun Xuanjing Huang Tat-Seng Chua Tianwei Zhang ALM ELM 96 2 0 26 Apr 2025
CURIE: Evaluating LLMs On Multitask Scientific Long Context Understanding and Reasoning Hao Cui Zahra Shamsi Gowoon Cheon Xuejian Ma Shutong Li ... Eun-Ah Kim M. Brenner Viren Jain Sameera Ponda Subhashini Venugopalan ELM LRM 57 0 0 14 Mar 2025
HierarQ: Task-Aware Hierarchical Q-Former for Enhanced Video Understanding Shehreen Azad Vibhav Vineet Yogesh S Rawat VLM 205 1 0 11 Mar 2025
Shifting Long-Context LLMs Research from Input to Output Yuhao Wu Yushi Bai Zhiqing Hu Shangqing Tu Ming Shan Hee Juanzi Li Roy Ka-Wei Lee 65 1 0 06 Mar 2025
LCIRC: A Recurrent Compression Approach for Efficient Long-form Context and Query Dependent Modeling in LLMs Sumin An Junyoung Sung Wonpyo Park Chanjun Park Paul Hongsuck Seo 102 0 0 10 Feb 2025
Can LLMs Maintain Fundamental Abilities under KV Cache Compression? Xiang Liu Zhenheng Tang Hong Chen Peijie Dong Zeyu Li Xiuze Zhou Bo Li Xuming Hu Xiaowen Chu 242 4 0 04 Feb 2025
LongReason: A Synthetic Long-Context Reasoning Benchmark via Context Expansion Zhan Ling Kang Liu Kai Yan Yuqing Yang Weijian Lin Ting-Han Fan Lingfeng Shen Zhengyin Du Jiecao Chen ReLM ELM LRM 52 3 0 25 Jan 2025
ELITR-Bench: A Meeting Assistant Benchmark for Long-Context Language Models Thibaut Thonet Jos Rozen Laurent Besacier RALM 145 2 0 20 Jan 2025
VisDoM: Multi-Document QA with Visually Rich Elements Using Multimodal Retrieval-Augmented Generation Manan Suri Puneet Mathur Franck Dernoncourt Kanika Goswami Ryan Rossi Dinesh Manocha 102 3 0 14 Dec 2024
Needle Threading: Can LLMs Follow Threads through Near-Million-Scale Haystacks? Jonathan Roberts Kai Han Samuel Albanie LLMAG 215 0 0 07 Nov 2024
Guide-LLM: An Embodied LLM Agent and Text-Based Topological Map for Robotic Guidance of People with Visual Impairments Sangmim Song S. Kodagoda A. Gunatilake Marc G. Carmichael Karthick Thiyagarajan Jodi Martin LM&Ro 30 1 0 28 Oct 2024
ETHIC: Evaluating Large Language Models on Long-Context Tasks with High Information Coverage Taewhoo Lee Chanwoong Yoon Kyochul Jang Donghyeon Lee Minju Song Hyunjae Kim Jaewoo Kang ELM 35 1 0 22 Oct 2024
LongGenBench: Long-context Generation Benchmark Xiang Liu Peijie Dong Xuming Hu Xiaowen Chu RALM 55 8 0 05 Oct 2024
HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughly Howard Yen Tianyu Gao Minmin Hou Ke Ding Daniel Fleischer Peter Izsak Moshe Wasserblat Danqi Chen ALM ELM 62 26 0 03 Oct 2024
How to Train Long-Context Language Models (Effectively) Tianyu Gao Alexander Wettig Howard Yen Danqi Chen RALM 72 39 0 03 Oct 2024
Locret: Enhancing Eviction in Long-Context LLM Inference with Trained Retaining Heads on Consumer-Grade Devices Yuxiang Huang Binhang Yuan Xu Han Chaojun Xiao Zhiyuan Liu RALM 87 1 0 02 Oct 2024
Bridging Context Gaps: Leveraging Coreference Resolution for Long Contextual Understanding Yanming Liu Xinyue Peng Jiannan Cao Shi Bo Yanxin Shen Tianyu Du Sheng Cheng Xun Wang Jianwei Yin Xuhong Zhang 71 9 0 02 Oct 2024
Summary of a Haystack: A Challenge to Long-Context LLMs and RAG Systems Philippe Laban Alexander R. Fabbri Caiming Xiong Chien-Sheng Wu RALM 51 41 0 01 Jul 2024
Leave No Document Behind: Benchmarking Long-Context LLMs with Extended Multi-Doc QA Minzheng Wang Longze Chen Cheng Fu Shengyi Liao Xinghua Zhang ... Run Luo Yunshui Li Min Yang Fei Huang Yongbin Li RALM 54 44 0 25 Jun 2024
BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack Yuri Kuratov Aydar Bulatov Petr Anokhin Ivan Rodkin Dmitry Sorokin Artyom Sorokin Andrey Kravchenko RALM ALM LRM ReLM ELM 51 61 0 14 Jun 2024
CS-Bench: A Comprehensive Benchmark for Large Language Models towards Computer Science Mastery Xiaoshuai Song Muxi Diao Guanting Dong Zhengyang Wang Yujia Fu ... Yejie Wang Zhuoma Gongque Jianing Yu Qiuna Tan Weiran Xu ELM 57 11 0 12 Jun 2024
Analyzing Temporal Complex Events with Large Language Models? A Benchmark towards Temporal, Long Context Understanding Zhihan Zhang Yixin Cao Chenchen Ye Yunshan Ma Lizi Liao Tat-Seng Chua 44 9 0 04 Jun 2024
PyramidInfer: Pyramid KV Cache Compression for High-throughput LLM Inference Dongjie Yang Xiaodong Han Yan Gao Yao Hu Shilin Zhang Hai Zhao 41 53 0 21 May 2024
Training-Free Long-Context Scaling of Large Language Models Chen An Fei Huang Jun Zhang Shansan Gong Xipeng Qiu Chang Zhou Lingpeng Kong ALM LRM 40 35 0 27 Feb 2024
On the Efficacy of Eviction Policy for Key-Value Constrained Generative Language Model Inference Siyu Ren Kenny Q. Zhu 26 27 0 09 Feb 2024
Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs Simone Balloccu Patrícia Schmidtová Mateusz Lango Ondrej Dusek SILM ELM PILM 35 159 0 06 Feb 2024
PROXYQA: An Alternative Framework for Evaluating Long-Form Text Generation with Large Language Models Haochen Tan Zhijiang Guo Zhan Shi Lu Xu Zhili Liu ... Xiaoguang Li Yasheng Wang Lifeng Shang Qun Liu Linqi Song 46 12 0 26 Jan 2024
Zebra: Extending Context Window with Layerwise Grouped Local-Global Attention Kaiqiang Song Xiaoyang Wang Sangwoo Cho Xiaoman Pan Dong Yu 36 7 0 14 Dec 2023
Recursively Summarizing Enables Long-Term Dialogue Memory in Large Language Models Qingyue Wang Y. Fu Yanan Cao Zhiliang Tian Shi Wang Dacheng Tao LLMAG KELM RALM 70 24 0 29 Aug 2023
Unlimiformer: Long-Range Transformers with Unlimited Length Input Amanda Bertsch Uri Alon Graham Neubig Matthew R. Gormley RALM 116 122 0 02 May 2023
SQuALITY: Building a Long-Document Summarization Dataset the Hard Way Alex Jinpeng Wang Richard Yuanzhe Pang Angelica Chen Jason Phang Samuel R. Bowman 78 44 0 23 May 2022
MultiDoc2Dial: Modeling Dialogues Grounded in Multiple Documents Song Feng S. Patel H. Wan Sachindra Joshi 59 66 0 26 Sep 2021
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 253 701 0 27 Aug 2021
Can We Automate Scientific Reviewing? Weizhe Yuan Pengfei Liu Graham Neubig 83 84 0 30 Jan 2021