Forgetting Curve: A Reliable Method for Evaluating Memorization
Capability for Long-context Models

Forgetting Curve: A Reliable Method for Evaluating Memorization Capability for Long-context Models

7 October 2024

Xinyu Liu

Runsong Zhao

Pengcheng Huang

Bei Li

Jingang Wang

Jingbo Zhu

ArXiv (abs)PDF HTML Github (20★)

Papers citing "Forgetting Curve: A Reliable Method for Evaluating Memorization Capability for Long-context Models"

9 / 9 papers shown

Title
Can Perplexity Reflect Large Language Model's Ability in Long Text Understanding? Yutong Hu Quzhe Huang Mingxu Tao Chen Zhang Yansong Feng 73 31 0 09 May 2024
$$\infty$Bench: Extending Long Context Evaluation Beyond 100K Tokens$ $\infty$ Bench: Extending Long Context Evaluation Beyond 100K Tokens Xinrong Zhang Yingfa Chen Shengding Hu Zihang Xu Junhao Chen ... Xu Han Zhen Leng Thai Shuo Wang Zhiyuan Liu Maosong Sun RALM LRM 98 193 0 21 Feb 2024
LV-Eval: A Balanced Long-Context Benchmark with 5 Length Levels Up to 256K Tao Yuan Xuefei Ning Dong Zhou Zhijie Yang Shiyao Li ... Dahua Lin Boxun Li Guohao Dai Shengen Yan Yu Wang ALM 86 46 0 06 Feb 2024
LooGLE: Can Long-Context Language Models Understand Long Contexts? Jiaqi Li Mengmeng Wang Zilong Zheng Muhan Zhang ELM RALM 77 133 0 08 Nov 2023
RWKV: Reinventing RNNs for the Transformer Era Bo Peng Eric Alcaide Quentin G. Anthony Alon Albalak Samuel Arcadinho ... Qihang Zhao P. Zhou Qinghua Zhou Jian Zhu Rui-Jie Zhu 233 607 0 22 May 2023
RoFormer: Enhanced Transformer with Rotary Position Embedding Jianlin Su Yu Lu Shengfeng Pan Ahmed Murtadha Bo Wen Yunfeng Liu 284 2,521 0 20 Apr 2021
Long Range Arena: A Benchmark for Efficient Transformers Yi Tay Mostafa Dehghani Samira Abnar Songlin Yang Dara Bahri Philip Pham J. Rao Liu Yang Sebastian Ruder Donald Metzler 152 727 0 08 Nov 2020
ELI5: Long Form Question Answering Angela Fan Yacine Jernite Ethan Perez David Grangier Jason Weston Michael Auli AI4MH ELM 103 624 0 22 Jul 2019
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context Zihang Dai Zhilin Yang Yiming Yang J. Carbonell Quoc V. Le Ruslan Salakhutdinov VLM 253 3,745 0 09 Jan 2019