Counterfactual Memorization in Neural Language Models

24 December 2021

Papers citing "Counterfactual Memorization in Neural Language Models"

28 / 28 papers shown

Title
On the Generalization Properties of Diffusion Models Puheng Li Zhong Li Huishuai Zhang Jiang Bian 74 29 0 13 Mar 2025
Obliviate: Efficient Unmemorization for Protecting Intellectual Property in Large Language Models M. Russinovich Ahmed Salem MU CLL 65 0 0 20 Feb 2025
SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training Tianzhe Chu Yuexiang Zhai Jihan Yang Shengbang Tong Saining Xie Dale Schuurmans Quoc V. Le Sergey Levine Yi Ma OffRL 70 58 0 28 Jan 2025
Impeding LLM-assisted Cheating in Introductory Programming Assignments via Adversarial Perturbation Saiful Islam Salim Rubin Yuchan Yang Alexander Cooper Suryashree Ray Saumya Debray Sazzadur Rahaman AAML 47 0 0 12 Oct 2024
Undesirable Memorization in Large Language Models: A Survey Ali Satvaty Suzan Verberne Fatih Turkmen ELM PILM 74 7 0 03 Oct 2024
Reasoning Elicitation in Language Models via Counterfactual Feedback Alihan Hüyük Xinnuo Xu Jacqueline Maasch Aditya V. Nori Javier González ReLM LRM 151 1 0 02 Oct 2024
Strong Copyright Protection for Language Models via Adaptive Model Fusion Javier Abad Konstantin Donhauser Francesco Pinto Fanny Yang 45 4 0 29 Jul 2024
Recite, Reconstruct, Recollect: Memorization in LMs as a Multifaceted Phenomenon USVSN Sai Prashanth Alvin Deng Kyle O'Brien Jyothir S V Mohammad Aflah Khan ... Jacob Ray Fuehne Stella Biderman Tracy Ke Katherine Lee Naomi Saphra 60 12 0 25 Jun 2024
REVS: Unlearning Sensitive Information in Language Models via Rank Editing in the Vocabulary Space Tomer Ashuach Martin Tutek Yonatan Belinkov KELM MU 71 4 0 13 Jun 2024
Quantifying In-Context Reasoning Effects and Memorization Effects in LLMs Siyu Lou Yuntian Chen Xiaodan Liang Liang Lin Quanshi Zhang 37 2 0 20 May 2024
To Each (Textual Sequence) Its Own: Improving Memorized-Data Unlearning in Large Language Models George-Octavian Barbulescu Peter Triantafillou MU 36 16 0 06 May 2024
The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG) Shenglai Zeng Jiankun Zhang Pengfei He Yue Xing Yiding Liu ... Jie Ren Shuaiqiang Wang Dawei Yin Yi Chang Jiliang Tang SILM 38 67 0 23 Feb 2024
Quantifying and Analyzing Entity-level Memorization in Large Language Models Zhenhong Zhou Jiuyang Xiang Chao-Yi Chen Sen Su PILM 38 8 0 30 Aug 2023
Selective Pre-training for Private Fine-tuning Da Yu Sivakanth Gopi Janardhan Kulkarni Zinan Lin Saurabh Naik Tomasz Religa Jian Yin Huishuai Zhang 35 19 0 23 May 2023
PreCog: Exploring the Relation between Memorization and Performance in Pre-trained Language Models Leonardo Ranaldi Elena Sofia Ruzzetti Fabio Massimo Zanzotto 31 6 0 08 May 2023
Recognition, recall, and retention of few-shot memories in large language models A. Orhan LRM KELM CLL 39 3 0 30 Mar 2023
Finding Memo: Extractive Memorization in Constrained Sequence Generation Tasks Vikas Raunak Arul Menezes 38 13 0 24 Oct 2022
Understanding Transformer Memorization Recall Through Idioms Adi Haviv Ido Cohen Jacob Gidron R. Schuster Yoav Goldberg Mor Geva 28 48 0 07 Oct 2022
GLM-130B: An Open Bilingual Pre-trained Model Aohan Zeng Xiao Liu Zhengxiao Du Zihan Wang Hanyu Lai ... Jidong Zhai Wenguang Chen Peng-Zhen Zhang Yuxiao Dong Jie Tang BDL LRM 250 1,073 0 05 Oct 2022
The Privacy Onion Effect: Memorization is Relative Nicholas Carlini Matthew Jagielski Chiyuan Zhang Nicolas Papernot Andreas Terzis Florian Tramèr PILM MIACV 33 99 0 21 Jun 2022
Memorization in NLP Fine-tuning Methods Fatemehsadat Mireshghallah Archit Uniyal Tianhao Wang David E. Evans Taylor Berg-Kirkpatrick AAML 61 39 0 25 May 2022
Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models Kushal Tirumala Aram H. Markosyan Luke Zettlemoyer Armen Aghajanyan TDI 29 185 0 22 May 2022
GPT-NeoX-20B: An Open-Source Autoregressive Language Model Sid Black Stella Biderman Eric Hallahan Quentin G. Anthony Leo Gao ... Shivanshu Purohit Laria Reynolds J. Tow Benqi Wang Samuel Weinbach 78 801 0 14 Apr 2022
Do Language Models Plagiarize? Jooyoung Lee Thai Le Jinghui Chen Dongwon Lee 36 74 0 15 Mar 2022
Deduplicating Training Data Mitigates Privacy Risks in Language Models Nikhil Kandpal Eric Wallace Colin Raffel PILM MU 45 274 0 14 Feb 2022
ModelPred: A Framework for Predicting Trained Model from Training Data Yingyan Zeng Jiachen T. Wang Si-An Chen H. Just Ran Jin R. Jia TDI MU 33 2 0 24 Nov 2021
Deduplicating Training Data Makes Language Models Better Katherine Lee Daphne Ippolito A. Nystrom Chiyuan Zhang Douglas Eck Chris Callison-Burch Nicholas Carlini SyDa 242 593 0 14 Jul 2021
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown D. Song Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 290 1,815 0 14 Dec 2020