Public Data-Assisted Mirror Descent for Private Model Training

1 December 2021

Ehsan Amid

Arun Ganesh

Rajiv Mathews

Swaroop Indra Ramaswamy

Papers citing "Public Data-Assisted Mirror Descent for Private Model Training"

40 / 40 papers shown

Title
FedDuA: Doubly Adaptive Federated Learning Shokichi Takakura Seng Pei Liew Satoshi Hasegawa FedML 19 0 0 16 May 2025
A mean teacher algorithm for unlearning of language models Yegor Klochkov MU 165 0 0 18 Apr 2025
Training Large ASR Encoders with Differential Privacy Geeticka Chauhan Steve Chien Om Thakkar Abhradeep Thakurta Arun Narayanan 33 1 0 21 Sep 2024
Private Geometric Median Mahdi Haghifam Thomas Steinke Jonathan R. Ullman 35 0 0 11 Jun 2024
HRNet: Differentially Private Hierarchical and Multi-Resolution Network for Human Mobility Data Synthesization Shun Takagi Li Xiong Fumiyuki Kato Yang Cao Masatoshi Yoshikawa 3DH 46 2 0 13 May 2024
Public-data Assisted Private Stochastic Optimization: Power and Limitations Enayat Ullah Michael Menart Raef Bassily Cristóbal Guzmán Raman Arora 30 1 0 06 Mar 2024
Pre-training Differentially Private Models with Limited Public Data Zhiqi Bu Xinwei Zhang Mingyi Hong Sheng Zha George Karypis 79 3 0 28 Feb 2024
How to Make the Gradients Small Privately: Improved Rates for Differentially Private Non-Convex Optimization Andrew Lowy Jonathan R. Ullman Stephen J. Wright 43 6 0 17 Feb 2024
Oracle-Efficient Differentially Private Learning with Public Data Adam Block Mark Bun Rathin Desai Abhishek Shetty Steven Wu FedML 24 2 0 13 Feb 2024
On the Benefits of Public Representations for Private Transfer Learning under Distribution Shift Pratiksha Thaker Amrith Rajagopal Setlur Zhiwei Steven Wu Virginia Smith 44 2 0 24 Dec 2023
Private Learning with Public Features Walid Krichene Nicolas Mayoraz Steffen Rendle Shuang Song Abhradeep Thakurta Li Zhang 27 6 0 24 Oct 2023
Mean Estimation Under Heterogeneous Privacy Demands Syomantak Chaudhuri Konstantin Miagkov T. Courtade 14 1 0 19 Oct 2023
Coupling public and private gradient provably helps optimization Ruixuan Liu Zhiqi Bu Yu-Xiang Wang Sheng Zha George Karypis 39 2 0 02 Oct 2023
Private Matrix Factorization with Public Item Features Mihaela Curmei Walid Krichene Li Zhang Mukund Sundararajan 37 3 0 17 Sep 2023
Private Distribution Learning with Public Data: The View from Sample Compression Shai Ben-David Alex Bie C. Canonne Gautam Kamath Vikrant Singhal 42 11 0 11 Aug 2023
Client-Level Differential Privacy via Adaptive Intermediary in Federated Medical Imaging Meirui Jiang Yuan Zhong Anjie Le Xiaoxiao Li Qianming Dou FedML 50 5 0 24 Jul 2023
Differentially Private Image Classification by Learning Priors from Random Processes Xinyu Tang Ashwinee Panda Vikash Sehwag Prateek Mittal 23 20 0 08 Jun 2023
PILLAR: How to make semi-private learning more effective Francesco Pinto Yaxian Hu Fanny Yang Amartya Sanyal 52 11 0 06 Jun 2023
Selective Pre-training for Private Fine-tuning Da Yu Sivakanth Gopi Janardhan Kulkarni Zinan Lin Saurabh Naik Tomasz Religa Jian Yin Huishuai Zhang 38 19 0 23 May 2023
Can Public Large Language Models Help Private Cross-device Federated Learning? Wei Ping Yibo Jacky Zhang Yuan Cao Bo-wen Li H. B. McMahan Sewoong Oh Zheng Xu Manzil Zaheer FedML 29 37 0 20 May 2023
Mean Estimation Under Heterogeneous Privacy: Some Privacy Can Be Free Syomantak Chaudhuri T. Courtade 30 4 0 27 Apr 2023
Differentially Private Stochastic Convex Optimization in (Non)-Euclidean Space Revisited Jinyan Su Changhong Zhao Di Wang 33 3 0 31 Mar 2023
How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy Natalia Ponomareva Hussein Hazimeh Alexey Kurakin Zheng Xu Carson E. Denison H. B. McMahan Sergei Vassilvitskii Steve Chien Abhradeep Thakurta 96 167 0 01 Mar 2023
Why Is Public Pretraining Necessary for Private Model Training? Arun Ganesh Mahdi Haghifam Milad Nasr Sewoong Oh Thomas Steinke Om Thakkar Abhradeep Thakurta Lun Wang 26 36 0 19 Feb 2023
Differentially Private Natural Language Models: Recent Advances and Future Directions Lijie Hu Ivan Habernal Lei Shen Di Wang AAML 30 18 0 22 Jan 2023
A New Linear Scaling Rule for Private Adaptive Hyperparameter Optimization Ashwinee Panda Xinyu Tang Saeed Mahloujifar Vikash Sehwag Prateek Mittal 43 11 0 08 Dec 2022
Differentially Private Adaptive Optimization with Delayed Preconditioners Tian Li Manzil Zaheer Ziyu Liu Sashank J. Reddi H. B. McMahan Virginia Smith 47 10 0 01 Dec 2022
Learning-Augmented Private Algorithms for Multiple Quantile Release M. Khodak Kareem Amin Travis Dick Sergei Vassilvitskii FedML 29 4 0 20 Oct 2022
Recycling Scraps: Improving Private Learning by Leveraging Intermediate Checkpoints Virat Shejwalkar Arun Ganesh Rajiv Mathews Om Thakkar Abhradeep Thakurta 34 8 0 04 Oct 2022
Private Estimation with Public Data Alex Bie Gautam Kamath Vikrant Singhal 36 28 0 16 Aug 2022
FLAIR: Federated Learning Annotated Image Repository Congzheng Song Filip Granqvist Kunal Talwar FedML 26 28 0 18 Jul 2022
When Does Differentially Private Learning Not Suffer in High Dimensions? Xuechen Li Daogao Liu Tatsunori Hashimoto Huseyin A. Inan Janardhan Kulkarni Y. Lee Abhradeep Thakurta 34 58 0 01 Jul 2022
Mixed Federated Learning: Joint Decentralized and Centralized Learning S. Augenstein Andrew Straiton Hard Lin Ning K. Singhal Satyen Kale Kurt Partridge Rajiv Mathews FedML 38 8 0 26 May 2022
Private Adaptive Optimization with Side Information Tian Li Manzil Zaheer Sashank J. Reddi Virginia Smith 37 35 0 12 Feb 2022
Federated Learning with Heterogeneous Differential Privacy Nasser Aldaghri Hessam Mahdavifar Ahmad Beirami FedML 35 2 0 28 Oct 2021
Differentially Private Fine-tuning of Language Models Da Yu Saurabh Naik A. Backurs Sivakanth Gopi Huseyin A. Inan ... Y. Lee Andre Manoel Lukas Wutschitz Sergey Yekhanin Huishuai Zhang 134 347 0 13 Oct 2021
Practical and Private (Deep) Learning without Sampling or Shuffling Peter Kairouz Brendan McMahan Shuang Song Om Thakkar Abhradeep Thakurta Zheng Xu FedML 182 154 0 26 Feb 2021
Do Not Let Privacy Overbill Utility: Gradient Embedding Perturbation for Private Learning Da Yu Huishuai Zhang Wei Chen Tie-Yan Liu FedML SILM 94 110 0 25 Feb 2021
Leveraging Public Data for Practical Private Query Release Terrance Liu G. Vietri Thomas Steinke Jonathan R. Ullman Zhiwei Steven Wu 161 58 0 17 Feb 2021
Tempered Sigmoid Activations for Deep Learning with Differential Privacy Nicolas Papernot Abhradeep Thakurta Shuang Song Steve Chien Ulfar Erlingsson AAML 147 178 0 28 Jul 2020