Do Membership Inference Attacks Work on Large Language Models?

12 February 2024

Michael Duan

Anshuman Suri

Niloofar Mireshghallah

Sewon Min

Weijia Shi

Luke Zettlemoyer

Yejin Choi

Papers citing "Do Membership Inference Attacks Work on Large Language Models?"

20 / 20 papers shown

Title
WaterDrum: Watermarking for Data-centric Unlearning Metric Xinyang Lu Xinyuan Niu Gregory Kang Ruey Lau Bui Thi Cam Nhung Rachael Hwee Ling Sim Fanyu Wen Chuan-Sheng Foo Szu Hui Ng Bryan Kian Hsiang Low MU 59 0 0 08 May 2025
A Survey on Privacy Risks and Protection in Large Language Models Kang Chen Xiuze Zhou Yuanguo Lin Shibo Feng Li Shen Pengcheng Wu AILaw PILM 147 0 0 04 May 2025
Beyond Public Access in LLM Pre-Training Data Sruly Rosenblat Tim O'Reilly Ilan Strauss MLAU 57 0 0 24 Apr 2025
Measuring Déjà vu Memorization Efficiently Narine Kokhlikyan Bargav Jayaraman Florian Bordes Chuan Guo Kamalika Chaudhuri 30 1 0 08 Apr 2025
Synthetic Data Can Mislead Evaluations: Membership Inference as Machine Text Detection Ali Naseh Niloofar Mireshghallah 58 0 0 20 Jan 2025
Has an AI model been trained on your images? Matyáš Boháček Hany Farid 38 0 0 11 Jan 2025
Detecting Training Data of Large Language Models via Expectation Maximization Gyuwan Kim Yang Li Evangelia Spiliopoulou Jie Ma Miguel Ballesteros William Yang Wang MIALM 95 4 2 10 Oct 2024
Fine-tuning can Help Detect Pretraining Data from Large Language Models H. Zhang Songxin Zhang Bingyi Jing Hongxin Wei 43 0 0 09 Oct 2024
Ward: Provable RAG Dataset Inference via LLM Watermarks Nikola Jovanović Robin Staab Maximilian Baader Martin Vechev 145 1 0 04 Oct 2024
Membership Inference Attacks Cannot Prove that a Model Was Trained On Your Data Jie Zhang Debeshee Das Gautam Kamath Florian Tramèr MIALM MIACV 235 16 1 29 Sep 2024
Generated Data with Fake Privacy: Hidden Dangers of Fine-tuning Large Language Models on Generated Data Atilla Akkus Mingjie Li Junjie Chu Junjie Chu Michael Backes Sinem Sav Sinem Sav SILM SyDa 45 1 0 12 Sep 2024
Range Membership Inference Attacks Jiashu Tao Reza Shokri 42 1 0 09 Aug 2024
Exposing Privacy Gaps: Membership Inference Attack on Preference Data for LLM Alignment Qizhang Feng Siva Rajesh Kasa Santhosh Kumar Kasa Hyokun Yun C. Teo S. Bodapati 92 6 0 08 Jul 2024
Blind Baselines Beat Membership Inference Attacks for Foundation Models Debeshee Das Jie Zhang Florian Tramèr MIALM 85 28 1 23 Jun 2024
The Mosaic Memory of Large Language Models Igor Shilov Matthieu Meeus Yves-Alexandre de Montjoye 44 3 0 24 May 2024
Min-K%++: Improved Baseline for Detecting Pre-Training Data from Large Language Models Jingyang Zhang Jingwei Sun Eric C. Yeats Ouyang Yang Martin Kuo Jianyi Zhang Hao Frank Yang Hai "Helen" Li 43 41 0 03 Apr 2024
Alpaca against Vicuna: Using LLMs to Uncover Memorization of LLMs Aly M. Kassem Omar Mahmoud Niloofar Mireshghallah Hyunwoo J. Kim Yulia Tsvetkov Yejin Choi Sherif Saad Santu Rana 50 18 0 05 Mar 2024
Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4 Kent K. Chang Mackenzie Cramer Sandeep Soni David Bamman RALM 145 111 0 28 Apr 2023
Deduplicating Training Data Makes Language Models Better Katherine Lee Daphne Ippolito A. Nystrom Chiyuan Zhang Douglas Eck Chris Callison-Burch Nicholas Carlini SyDa 242 592 0 14 Jul 2021
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown D. Song Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 290 1,815 0 14 Dec 2020