Investigating Data Contamination for Pre-training Language Models

Investigating Data Contamination for Pre-training Language Models

11 January 2024

Rylan Schaeffer

Papers citing "Investigating Data Contamination for Pre-training Language Models"

17 / 17 papers shown

Title
How Can I Publish My LLM Benchmark Without Giving the True Answers Away? Takashi Ishida Thanawat Lodkaew Ikko Yamane 137 0 0 23 May 2025
Preference Leakage: A Contamination Problem in LLM-as-a-judge Dawei Li Renliang Sun Yue Huang Ming Zhong Bohan Jiang Jiawei Han Wei Wei Wei Wang Huan Liu 99 23 0 03 Feb 2025
SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training Tianzhe Chu Yuexiang Zhai Jihan Yang Shengbang Tong Saining Xie Dale Schuurmans Quoc V. Le Sergey Levine Yi-An Ma OffRL 123 85 0 28 Jan 2025
AntiLeakBench: Preventing Data Contamination by Automatically Constructing Benchmarks with Updated Real-World Knowledge Xiaobao Wu Liangming Pan Yuxi Xie Ruiwen Zhou Shuai Zhao Yubo Ma Mingzhe Du Rui Mao Anh Tuan Luu William Yang Wang 189 12 0 18 Dec 2024
LLM-as-an-Interviewer: Beyond Static Testing Through Dynamic LLM Evaluation Eunsu Kim Juyoung Suk Seungone Kim Niklas Muennighoff Dongkwan Kim Alice Oh ELM 115 1 0 10 Dec 2024
Improving Model Evaluation using SMART Filtering of Benchmark Datasets Vipul Gupta Candace Ross David Pantoja R. Passonneau Megan Ung Adina Williams 218 2 0 26 Oct 2024
How Much Can We Forget about Data Contamination? Sebastian Bordt Suraj Srinivas Valentyn Boreiko U. V. Luxburg 89 2 0 04 Oct 2024
ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilities Ezra Karger Houtan Bastani Chen Yueh-Han Zachary Jacobs Danny Halawi Fred Zhang P. Tetlock 87 7 0 30 Sep 2024
Bilingual Evaluation of Language Models on General Knowledge in University Entrance Exams with Minimal Contamination Eva Sánchez Salido Roser Morante Julio Gonzalo Guillermo Marco Jorge Carrillo-de-Albornoz ... Enrique Amigó Andrés Fernández Alejandro Benito-Santos Adrián Ghajari Espinosa Victor Fresno ELM 73 0 0 19 Sep 2024
Language Models Trained to do Arithmetic Predict Human Risky and Intertemporal Choice Jian-Qiao Zhu Haijiang Yan Thomas Griffiths 93 2 0 29 May 2024
Hatred Stems from Ignorance! Distillation of the Persuasion Modes in Countering Conversational Hate Speech Ghadi Alyahya Abeer Aldayel 62 2 0 18 Mar 2024
Rethinking Benchmark and Contamination for Language Models with Rephrased Samples Shuo Yang Wei-Lin Chiang Lianmin Zheng Joseph E. Gonzalez Ion Stoica ALM 51 122 0 08 Nov 2023
Detecting Pretraining Data from Large Language Models Weijia Shi Anirudh Ajith Mengzhou Xia Yangsibo Huang Daogao Liu Terra Blevins Danqi Chen Luke Zettlemoyer MIALM 59 177 0 25 Oct 2023
Towards a Unified Multi-Dimensional Evaluator for Text Generation Ming Zhong Yang Liu Da Yin Yuning Mao Yizhu Jiao Peng Liu Chenguang Zhu Heng Ji Jiawei Han ELM 64 263 0 13 Oct 2022
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 409 2,051 0 31 Dec 2020
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown D. Song Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 422 1,868 0 14 Dec 2020
SQuAD: 100,000+ Questions for Machine Comprehension of Text Pranav Rajpurkar Jian Zhang Konstantin Lopyrev Percy Liang RALM 186 8,067 0 16 Jun 2016