How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVEN

18 November 2021

Papers citing "How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVEN"

28 / 28 papers shown

Title
Do Large Language Models know who did what to whom? Joseph M. Denning Xiaohan Bryor Snefjella Idan A. Blank 57 1 0 23 Apr 2025
Are formal and functional linguistic mechanisms dissociated in language models? Michael Hanna Sandro Pezzelle Yonatan Belinkov 47 0 0 14 Mar 2025
Protecting Users From Themselves: Safeguarding Contextual Privacy in Interactions with Conversational Agents Ivoline Ngong Swanand Kadhe Hao Wang K. Murugesan Justin D. Weisz Amit Dhurandhar K. Ramamurthy 49 2 0 22 Feb 2025
Is This Collection Worth My LLM's Time? Automatically Measuring Information Potential in Text Corpora Tristan Karch Luca Engel Philippe Schwaller Frédéric Kaplan 77 0 0 20 Feb 2025
Does Data Contamination Detection Work (Well) for LLMs? A Survey and Evaluation on Detection Assumptions Yujuan Fu Özlem Uzuner Meliha Yetisgen Fei Xia 59 3 0 24 Oct 2024
Evaluating Morphological Compositional Generalization in Large Language Models Mete Ismayilzada Defne Çirci Jonne Sälevä Hale Sirin Abdullatif Köksal Bhuwan Dhingra Antoine Bosselut Lonneke van der Plas Duygu Ataman 28 2 0 16 Oct 2024
Undesirable Memorization in Large Language Models: A Survey Ali Satvaty Suzan Verberne Fatih Turkmen ELM PILM 71 7 0 03 Oct 2024
LLM-based multi-agent poetry generation in non-cooperative environments Ran Zhang Steffen Eger LLMAG 31 5 0 05 Sep 2024
Quantifying In-Context Reasoning Effects and Memorization Effects in LLMs Siyu Lou Yuntian Chen Xiaodan Liang Liang Lin Quanshi Zhang 32 2 0 20 May 2024
LongStory: Coherent, Complete and Length Controlled Long story Generation Kyeongman Park Nakyeong Yang Kyomin Jung 35 4 0 26 Nov 2023
How Well Do Large Language Models Truly Ground? Hyunji Lee Se June Joo Chaeeun Kim Joel Jang Doyoung Kim Kyoung-Woon On Minjoon Seo HILM 25 6 0 15 Nov 2023
On Using Distribution-Based Compositionality Assessment to Evaluate Compositional Generalisation in Machine Translation Anssi Moisio Mathias Creutz M. Kurimo CoGe 12 1 0 14 Nov 2023
Generative Models as a Complex Systems Science: How can we make sense of large language model behavior? Ari Holtzman Peter West Luke Zettlemoyer AI4CE 30 13 0 31 Jul 2023
Towards Generating Functionally Correct Code Edits from Natural Language Issue Descriptions Sarah Fakhoury Saikat Chakraborty Madan Musuvathi Shuvendu K. Lahiri 38 21 0 07 Apr 2023
Recognition, recall, and retention of few-shot memories in large language models A. Orhan LRM KELM CLL 34 3 0 30 Mar 2023
On the Creativity of Large Language Models Giorgio Franceschelli Mirco Musolesi 72 51 0 27 Mar 2023
Dissociating language and thought in large language models Kyle Mahowald Anna A. Ivanova I. Blank Nancy Kanwisher J. Tenenbaum Evelina Fedorenko ELM ReLM 25 209 0 16 Jan 2023
Self-Repetition in Abstractive Neural Summarizers Nikita Salkar T. Trikalinos Byron C. Wallace A. Nenkova 8 10 0 14 Oct 2022
Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models Kushal Tirumala Aram H. Markosyan Luke Zettlemoyer Armen Aghajanyan TDI 26 185 0 22 May 2022
TemporalWiki: A Lifelong Benchmark for Training and Evaluating Ever-Evolving Language Models Joel Jang Seonghyeon Ye Changho Lee Sohee Yang Joongbo Shin Janghoon Han Gyeonghun Kim Minjoon Seo CLL KELM 27 91 0 29 Apr 2022
Evaluating Distributional Distortion in Neural Language Modeling Benjamin LeBrun Alessandro Sordoni Timothy J. O'Donnell 17 22 0 24 Mar 2022
Do Language Models Plagiarize? Jooyoung Lee Thai Le Jinghui Chen Dongwon Lee 25 73 0 15 Mar 2022
Deduplicating Training Data Mitigates Privacy Risks in Language Models Nikhil Kandpal Eric Wallace Colin Raffel PILM MU 28 274 0 14 Feb 2022
Frequency Effects on Syntactic Rule Learning in Transformers Jason W. Wei Dan Garrette Tal Linzen Ellie Pavlick 85 62 0 14 Sep 2021
Deduplicating Training Data Makes Language Models Better Katherine Lee Daphne Ippolito A. Nystrom Chiyuan Zhang Douglas Eck Chris Callison-Burch Nicholas Carlini SyDa 242 591 0 14 Jul 2021
On Compositional Generalization of Neural Machine Translation Yafu Li Yongjing Yin Yulong Chen Yue Zhang 153 44 0 31 May 2021
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown D. Song Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 290 1,814 0 14 Dec 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 231 4,460 0 23 Jan 2020