Fine-Tuning Large Language Models with User-Level Differential Privacy

Fine-Tuning Large Language Models with User-Level Differential Privacy

10 July 2024

Zachary Charles

Nicole Mitchell

Krishna Pillutla

Papers citing "Fine-Tuning Large Language Models with User-Level Differential Privacy"

9 / 9 papers shown

Title
A Hassle-free Algorithm for Private Learning in Practice: Don't Use Tree Aggregation, Use BLTs H. B. McMahan Zheng Xu Yanxiang Zhang FedML 37 5 0 16 Aug 2024
Mind the Privacy Unit! User-Level Differential Privacy for Language Model Fine-Tuning Lynn Chua Badih Ghazi Yangsibo Huang Pritish Kamath Ravi Kumar Daogao Liu Pasin Manurangsi Amer Sinha Chiyuan Zhang 24 11 0 20 Jun 2024
How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy Natalia Ponomareva Hussein Hazimeh Alexey Kurakin Zheng Xu Carson E. Denison H. B. McMahan Sergei Vassilvitskii Steve Chien Abhradeep Thakurta 94 167 0 01 Mar 2023
Towards Standardized Mobility Reports with User-Level Privacy Alexandra Kapp Saskia Nuñez von Voigt Helena Mihaljević Florian Tschorsch 34 2 0 19 Sep 2022
Deduplicating Training Data Makes Language Models Better Katherine Lee Daphne Ippolito A. Nystrom Chiyuan Zhang Douglas Eck Chris Callison-Burch Nicholas Carlini SyDa 242 592 0 14 Jul 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 280 3,848 0 18 Apr 2021
Practical and Private (Deep) Learning without Sampling or Shuffling Peter Kairouz Brendan McMahan Shuang Song Om Thakkar Abhradeep Thakurta Zheng Xu FedML 182 154 0 26 Feb 2021
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown D. Song Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 290 1,814 0 14 Dec 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 237 4,469 0 23 Jan 2020