Koala: An Index for Quantifying Overlaps with Pre-training Corpora

Koala: An Index for Quantifying Overlaps with Pre-training Corpora

26 March 2023

Gholamreza Haffari

Ehsan Shareghi

Papers citing "Koala: An Index for Quantifying Overlaps with Pre-training Corpora"

16 / 16 papers shown

Title
RICo: Refined In-Context Contribution for Automatic Instruction-Tuning Data Selection Yixin Yang Qingxiu Dong Linli Yao Fangwei Zhu Zhifang Sui 48 0 0 08 May 2025
D3: Diversity, Difficulty, and Dependability-Aware Data Selection for Sample-Efficient LLM Instruction Tuning Jia Zhang Chen-Xi Zhang Yong-Jin Liu Yi-Xuan Jin Xiao-Wen Yang Bo Zheng Yi Liu Lan-Zhe Guo 47 2 0 14 Mar 2025
Language Model Preference Evaluation with Multiple Weak Evaluators Zhengyu Hu Jieyu Zhang Zhihan Xiong Alexander Ratner Hui Xiong Ranjay Krishna 51 3 0 14 Oct 2024
Language model developers should report train-test overlap Andy K. Zhang Kevin Klyman Yifan Mai Yoav Levine Yian Zhang Rishi Bommasani Percy Liang VLM ELM 31 8 0 10 Oct 2024
ConStat: Performance-Based Contamination Detection in Large Language Models Jasper Dekoninck Mark Niklas Muller Martin Vechev 37 6 0 25 May 2024
Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning Ming Li Lichang Chen Jiuhai Chen Shwai He Jiuxiang Gu Dinesh Manocha 26 51 0 15 Feb 2024
Evading Data Contamination Detection for Language Models is (too) Easy Jasper Dekoninck Mark Niklas Muller Maximilian Baader Marc Fischer Martin Vechev 96 18 0 05 Feb 2024
Infini-gram: Scaling Unbounded n-gram Language Models to a Trillion Tokens Jiacheng Liu Sewon Min Luke Zettlemoyer Yejin Choi Hannaneh Hajishirzi 48 50 0 30 Jan 2024
MoDS: Model-oriented Data Selection for Instruction Tuning Qianlong Du Chengqing Zong Jiajun Zhang ALM 26 77 0 27 Nov 2023
Which Prompts Make The Difference? Data Prioritization For Efficient Human LLM Evaluation M. Boubdir Edward Kim B. Ermiş Marzieh Fadaee Sara Hooker ALM 31 18 0 22 Oct 2023
From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuning Ming Li Yong Zhang Zhitao Li Jiuhai Chen Lichang Chen Ning Cheng Jianzong Wang Dinesh Manocha Jing Xiao 38 170 0 23 Aug 2023
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 328 4,077 0 24 May 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 313 11,953 0 04 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 370 8,495 0 28 Jan 2022
Memorisation versus Generalisation in Pre-trained Language Models Michael Tänzer Sebastian Ruder Marek Rei 94 50 0 16 Apr 2021
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 256 1,996 0 31 Dec 2020