Why Is Public Pretraining Necessary for Private Model Training?

Why Is Public Pretraining Necessary for Private Model Training?

19 February 2023

Abhradeep Thakurta

Papers citing "Why Is Public Pretraining Necessary for Private Model Training?"

11 / 11 papers shown

Title
Training Large ASR Encoders with Differential Privacy Geeticka Chauhan Steve Chien Om Thakkar Abhradeep Thakurta Arun Narayanan 33 1 0 21 Sep 2024
Public-data Assisted Private Stochastic Optimization: Power and Limitations Enayat Ullah Michael Menart Raef Bassily Cristóbal Guzmán Raman Arora 30 1 0 06 Mar 2024
Private Fine-tuning of Large Language Models with Zeroth-order Optimization Xinyu Tang Ashwinee Panda Milad Nasr Saeed Mahloujifar Prateek Mittal 44 18 0 09 Jan 2024
Unintended Memorization in Large ASR Models, and How to Mitigate It Lun Wang Om Thakkar Rajiv Mathews 33 5 0 18 Oct 2023
Private Distribution Learning with Public Data: The View from Sample Compression Shai Ben-David Alex Bie C. Canonne Gautam Kamath Vikrant Singhal 35 11 0 11 Aug 2023
Differentially Private Synthetic Data via Foundation Model APIs 1: Images Zi-Han Lin Sivakanth Gopi Janardhan Kulkarni Harsha Nori Sergey Yekhanin 39 36 0 24 May 2023
Can Public Large Language Models Help Private Cross-device Federated Learning? Boxin Wang Yibo Zhang Yuan Cao Bo-wen Li H. B. McMahan Sewoong Oh Zheng Xu Manzil Zaheer FedML 24 37 0 20 May 2023
Private Convex Optimization via Exponential Mechanism Sivakanth Gopi Y. Lee Daogao Liu 81 52 0 01 Mar 2022
Differentially Private Fine-tuning of Language Models Da Yu Saurabh Naik A. Backurs Sivakanth Gopi Huseyin A. Inan ... Y. Lee Andre Manoel Lukas Wutschitz Sergey Yekhanin Huishuai Zhang 134 346 0 13 Oct 2021
Do Not Let Privacy Overbill Utility: Gradient Embedding Perturbation for Private Learning Da Yu Huishuai Zhang Wei Chen Tie-Yan Liu FedML SILM 94 110 0 25 Feb 2021
Leveraging Public Data for Practical Private Query Release Terrance Liu G. Vietri Thomas Steinke Jonathan R. Ullman Zhiwei Steven Wu 150 58 0 17 Feb 2021