Emergent and Predictable Memorization in Large Language Models

21 April 2023

Papers citing "Emergent and Predictable Memorization in Large Language Models"

44 / 94 papers shown

Title
Alpaca against Vicuna: Using LLMs to Uncover Memorization of LLMs Aly M. Kassem Omar Mahmoud Niloofar Mireshghallah Hyunwoo J. Kim Yulia Tsvetkov Yejin Choi Sherif Saad Santu Rana 50 19 0 05 Mar 2024
Teach LLMs to Phish: Stealing Private Information from Language Models Ashwinee Panda Christopher A. Choquette-Choo Zhengming Zhang Yaoqing Yang Prateek Mittal PILM 40 20 0 01 Mar 2024
Wisdom of the Silicon Crowd: LLM Ensemble Prediction Capabilities Rival Human Crowd Accuracy P. Schoenegger Indre Tuminauskaite Peter S. Park Rafael Valdece Sousa Bastos P. Tetlock 43 27 0 29 Feb 2024
tinyBenchmarks: evaluating LLMs with fewer examples Felipe Maia Polo Lucas Weber Leshem Choshen Yuekai Sun Gongjun Xu Mikhail Yurochkin ELM 32 77 0 22 Feb 2024
Dynamic Evaluation of Large Language Models by Meta Probing Agents Kaijie Zhu Jindong Wang Qinlin Zhao Ruochen Xu Xing Xie 50 31 0 21 Feb 2024
Regulating Large Language Models: A Roundtable Report Gabriel Nicholas Paul Friedl ELM AILaw 29 1 0 16 Feb 2024
AI-Augmented Predictions: LLM Assistants Improve Human Forecasting Accuracy P. Schoenegger Peter S. Park Ezra Karger P. Tetlock 45 14 0 12 Feb 2024
Do Membership Inference Attacks Work on Large Language Models? Michael Duan Anshuman Suri Niloofar Mireshghallah Sewon Min Weijia Shi Luke Zettlemoyer Yulia Tsvetkov Yejin Choi David E. Evans Hanna Hajishirzi MIALM 42 79 0 12 Feb 2024
Copyright Protection in Generative AI: A Technical Perspective Jie Ren Han Xu Pengfei He Yingqian Cui Shenglai Zeng ... Hongzhi Wen Jiayuan Ding Hui Liu Yi Chang Jiliang Tang DeLMO 31 32 0 04 Feb 2024
Code Simulation Challenges for Large Language Models Emanuele La Malfa Christoph Weinhuber Orazio Torre Fangru Lin Samuele Marro Anthony Cohn Nigel Shadbolt Michael Wooldridge LLMAG LRM 22 8 0 17 Jan 2024
Traces of Memorisation in Large Language Models for Code Ali Al-Kaswan M. Izadi A. van Deursen ELM 39 14 0 18 Dec 2023
LLM360: Towards Fully Transparent Open-Source LLMs Zhengzhong Liu Aurick Qiao W. Neiswanger Hongyi Wang Bowen Tan ... Zhiting Hu Mark Schulze Preslav Nakov Timothy Baldwin Eric P. Xing 49 70 0 11 Dec 2023
Heaps' Law in GPT-Neo Large Language Model Emulated Corpora Uyen Lai Gurjit S. Randhawa Paul Sheridan 21 0 0 10 Nov 2023
KITAB: Evaluating LLMs on Constraint Satisfaction for Information Retrieval Marah Abdin Suriya Gunasekar Varun Chandrasekaran Jerry Li Mert Yuksekgonul Rahee Peshawaria Ranjita Naik Besmira Nushi 64 12 0 24 Oct 2023
Experimental Narratives: A Comparison of Human Crowdsourced Storytelling and AI Storytelling Nina Beguš 33 20 0 19 Oct 2023
Large Language Model Prediction Capabilities: Evidence from a Real-World Forecasting Tournament P. Schoenegger Peter S. Park ELM AI4TS 28 14 0 17 Oct 2023
Data Contamination Through the Lens of Time Manley Roberts Himanshu Thakur Christine Herlihy Colin White Samuel Dooley 84 31 0 16 Oct 2023
User Inference Attacks on Large Language Models Nikhil Kandpal Krishna Pillutla Alina Oprea Peter Kairouz Christopher A. Choquette-Choo Zheng Xu SILM AAML 44 15 0 13 Oct 2023
Large Language Models Are Zero-Shot Time Series Forecasters Nate Gruver Marc Finzi Shikai Qiu Andrew Gordon Wilson AI4TS 33 322 0 11 Oct 2023
Exploring Memorization in Fine-tuned Language Models Shenglai Zeng Yaxin Li Jie Ren Yiding Liu Han Xu Pengfei He Yue Xing Shuaiqiang Wang Jiliang Tang Dawei Yin PILM 41 23 0 10 Oct 2023
What do larger image classifiers memorise? Michal Lukasik Vaishnavh Nagarajan A. S. Rawat A. Menon Sanjiv Kumar 38 5 0 09 Oct 2023
The Cost of Down-Scaling Language Models: Fact Recall Deteriorates before In-Context Learning Tian Jin Nolan Clement Xin Dong Vaishnavh Nagarajan Michael Carbin Jonathan Ragan-Kelley Gintare Karolina Dziugaite LRM 54 5 0 07 Oct 2023
Meta Semantic Template for Evaluation of Large Language Models Yachuan Liu Liang Chen Jindong Wang Qiaozhu Mei Xing Xie 22 0 0 01 Oct 2023
DyVal: Dynamic Evaluation of Large Language Models for Reasoning Tasks A. Maritan Jiaao Chen S. Dey Luca Schenato Diyi Yang Xing Xie ELM LRM 27 42 0 29 Sep 2023
Identifying and Mitigating Privacy Risks Stemming from Language Models: A Survey Victoria Smith Ali Shahin Shamsabadi Carolyn Ashurst Adrian Weller PILM 32 24 0 27 Sep 2023
Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Models Mert Yuksekgonul Varun Chandrasekaran Erik Jones Suriya Gunasekar Ranjita Naik Hamid Palangi Ece Kamar Besmira Nushi HILM 26 40 0 26 Sep 2023
Are Emergent Abilities in Large Language Models just In-Context Learning? Sheng Lu Irina Bigoulaeva Rachneet Sachdeva Harish Tayyar Madabushi Iryna Gurevych LRM ELM ReLM 54 93 0 04 Sep 2023
Efficient Benchmarking of Language Models Yotam Perlitz Elron Bandel Ariel Gera Ofir Arviv L. Ein-Dor Eyal Shnarch Noam Slonim Michal Shmueli-Scheuer Leshem Choshen ALM 24 24 0 22 Aug 2023
OctoPack: Instruction Tuning Code Large Language Models Niklas Muennighoff Qian Liu A. Zebaze Qinkai Zheng Binyuan Hui Terry Yue Zhuo Swayam Singh Xiangru Tang Leandro von Werra Shayne Longpre VLM ALM 71 119 0 14 Aug 2023
What can we learn from Data Leakage and Unlearning for Law? Jaydeep Borkar PILM MU 38 10 0 19 Jul 2023
Tools for Verifying Neural Models' Training Data Dami Choi Yonadav Shavit David Duvenaud MIALM 22 14 0 02 Jul 2023
PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts Kaijie Zhu Jindong Wang Jiaheng Zhou Zichen Wang Hao Chen ... Linyi Yang Weirong Ye Yue Zhang Neil Zhenqiang Gong Xingxu Xie SILM 50 144 0 07 Jun 2023
DeepfakeArt Challenge: A Benchmark Dataset for Generative AI Art Forgery and Data Poisoning Detection Hossein Aboutalebi Daniel Mao Rongqi Fan Carol Xu Chris He Alexander Wong AAML 20 8 0 02 Jun 2023
Large Language Models, scientific knowledge and factuality: A systematic analysis in antibiotic discovery Magdalena Wysocka Oskar Wysocki Maxime Delmas V. Mutel André Freitas LM&MA 35 6 0 28 May 2023
Scaling Data-Constrained Language Models Niklas Muennighoff Alexander M. Rush Boaz Barak Teven Le Scao Aleksandra Piktus Nouamane Tazi S. Pyysalo Thomas Wolf Colin Raffel ALM 38 200 0 25 May 2023
Training Data Extraction From Pre-trained Language Models: A Survey Shotaro Ishihara 29 46 0 25 May 2023
Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling Stella Biderman Hailey Schoelkopf Quentin G. Anthony Herbie Bradley Kyle O'Brien ... USVSN Sai Prashanth Edward Raff Aviya Skowron Lintang Sutawika Oskar van der Wal 36 1,178 0 03 Apr 2023
Hallucinations in Large Multilingual Translation Models Nuno M. Guerreiro Duarte M. Alves Jonas Waldendorf Barry Haddow Alexandra Birch Pierre Colombo André F.T. Martins VLM HILM LRM 35 141 0 28 Mar 2023
What Language Model to Train if You Have One Million GPU Hours? Teven Le Scao Thomas Wang Daniel Hesslow Lucile Saulnier Stas Bekman ... Lintang Sutawika Jaesung Tae Zheng-Xin Yong Julien Launay Iz Beltagy MoE AI4CE 230 103 0 27 Oct 2022
A Systematic Evaluation of Large Language Models of Code Frank F. Xu Uri Alon Graham Neubig Vincent J. Hellendoorn ELM ALM 204 631 0 26 Feb 2022
Deduplicating Training Data Makes Language Models Better Katherine Lee Daphne Ippolito A. Nystrom Chiyuan Zhang Douglas Eck Chris Callison-Burch Nicholas Carlini SyDa 242 593 0 14 Jul 2021
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 282 1,996 0 31 Dec 2020
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown D. Song Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 290 1,824 0 14 Dec 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 264 4,489 0 23 Jan 2020