Why Is Public Pretraining Necessary for Private Model Training?

Why Is Public Pretraining Necessary for Private Model Training?

19 February 2023

Abhradeep Thakurta

Papers citing "Why Is Public Pretraining Necessary for Private Model Training?"

13 / 13 papers shown

Title
Training Large ASR Encoders with Differential Privacy Geeticka Chauhan Steve Chien Om Thakkar Abhradeep Thakurta Arun Narayanan 33 1 0 21 Sep 2024
Public-data Assisted Private Stochastic Optimization: Power and Limitations Enayat Ullah Michael Menart Raef Bassily Cristóbal Guzmán Raman Arora 30 1 0 06 Mar 2024
Private Fine-tuning of Large Language Models with Zeroth-order Optimization Xinyu Tang Ashwinee Panda Milad Nasr Saeed Mahloujifar Prateek Mittal 47 18 0 09 Jan 2024
Unintended Memorization in Large ASR Models, and How to Mitigate It Lun Wang Om Thakkar Rajiv Mathews 38 5 0 18 Oct 2023
Private Distribution Learning with Public Data: The View from Sample Compression Shai Ben-David Alex Bie C. Canonne Gautam Kamath Vikrant Singhal 42 11 0 11 Aug 2023
Differentially Private Image Classification by Learning Priors from Random Processes Xinyu Tang Ashwinee Panda Vikash Sehwag Prateek Mittal 23 20 0 08 Jun 2023
Differentially Private Synthetic Data via Foundation Model APIs 1: Images Zinan Lin Sivakanth Gopi Janardhan Kulkarni Harsha Nori Sergey Yekhanin 41 36 0 24 May 2023
Selective Pre-training for Private Fine-tuning Da Yu Sivakanth Gopi Janardhan Kulkarni Zinan Lin Saurabh Naik Tomasz Religa Jian Yin Huishuai Zhang 35 19 0 23 May 2023
Can Public Large Language Models Help Private Cross-device Federated Learning? Wei Ping Yibo Jacky Zhang Yuan Cao Bo-wen Li H. B. McMahan Sewoong Oh Zheng Xu Manzil Zaheer FedML 29 37 0 20 May 2023
Private Convex Optimization via Exponential Mechanism Sivakanth Gopi Y. Lee Daogao Liu 83 52 0 01 Mar 2022
Differentially Private Fine-tuning of Language Models Da Yu Saurabh Naik A. Backurs Sivakanth Gopi Huseyin A. Inan ... Y. Lee Andre Manoel Lukas Wutschitz Sergey Yekhanin Huishuai Zhang 134 347 0 13 Oct 2021
Do Not Let Privacy Overbill Utility: Gradient Embedding Perturbation for Private Learning Da Yu Huishuai Zhang Wei Chen Tie-Yan Liu FedML SILM 94 110 0 25 Feb 2021
Leveraging Public Data for Practical Private Query Release Terrance Liu G. Vietri Thomas Steinke Jonathan R. Ullman Zhiwei Steven Wu 158 58 0 17 Feb 2021