v1v2v3v4 (latest)

When Does Differentially Private Learning Not Suffer in High Dimensions?

1 July 2022

Tatsunori Hashimoto

ArXiv (abs)PDF HTML Github (174★)

Papers citing "When Does Differentially Private Learning Not Suffer in High Dimensions?"

50 / 51 papers shown

Title
An Optimization Framework for Differentially Private Sparse Fine-Tuning Mehdi Makni Kayhan Behdin Gabriel Afriat Zheng Xu Sergei Vassilvitskii Natalia Ponomareva Hussein Hazimeh Rahul Mazumder 122 0 0 17 Mar 2025
Tokens for Learning, Tokens for Unlearning: Mitigating Membership Inference Attacks in Large Language Models via Dual-Purpose Training Toan Tran Ruixuan Liu Li Xiong MU 115 1 0 27 Feb 2025
DP-MemArc: Differential Privacy Transfer Learning for Memory Efficient Language Models Yanming Liu Xinyue Peng Yuwei Zhang Xiaolan Ke Songhang Deng ... Sheng Cheng Xun Wang Yuxiang Cai Tianyu Du Xuhong Zhang 183 1 0 21 Feb 2025
A Theoretical Survey on Foundation Models Shi Fu Yuzhu Chen Yingjie Wang Dacheng Tao 90 0 0 15 Oct 2024
DiSK: Differentially Private Optimizer with Simplified Kalman Filter for Noise Reduction Xinwei Zhang Zhiqi Bu Borja Balle Mingyi Hong Meisam Razaviyayn Vahab Mirrokni 146 2 0 04 Oct 2024
Adaptively Private Next-Token Prediction of Large Language Models James Flemings Meisam Razaviyayn Murali Annavaram 135 1 0 02 Oct 2024
Differentially Private Parameter-Efficient Fine-tuning for Large ASR Models Hongbin Liu Lun Wang Om Thakkar Abhradeep Thakurta Arun Narayanan 123 0 0 02 Oct 2024
Training Large ASR Encoders with Differential Privacy Geeticka Chauhan Steve Chien Om Thakkar Abhradeep Thakurta Arun Narayanan 100 1 0 21 Sep 2024
LLM-PBE: Assessing Data Privacy in Large Language Models Qinbin Li Junyuan Hong Chulin Xie Jeffrey Tan Rachel Xin ... Dan Hendrycks Zhangyang Wang Bo Li Bingsheng He Dawn Song ELM PILM 120 18 0 23 Aug 2024
Mitigating Noise Detriment in Differentially Private Federated Learning with Model Pre-training Huitong Jin Yipeng Zhou Laizhong Cui Quan Z. Sheng AI4CE 74 0 0 18 Aug 2024
PrE-Text: Training Language Models on Private Federated Data in the Age of LLMs Charlie Hou Akshat Shrivastava Hongyuan Zhan Rylan Conway Trang Le Adithya Sagar Giulia Fanti Daniel Lazar 118 15 0 05 Jun 2024
Neural Collapse Meets Differential Privacy: Curious Behaviors of NoisyGD with Near-perfect Representation Learning Chendi Wang Yuqing Zhu Weijie J. Su Yu Wang AAML 103 5 0 14 May 2024
Improving LoRA in Privacy-preserving Federated Learning Youbang Sun Zitao Li Yaliang Li Bolin Ding 91 81 0 18 Mar 2024
Differentially Private Representation Learning via Image Captioning Tom Sander Yaodong Yu Maziar Sanjabi Alain Durmus Yi-An Ma Kamalika Chaudhuri Chuan Guo 106 4 0 04 Mar 2024
Differentially Private Knowledge Distillation via Synthetic Text Generation James Flemings Murali Annavaram SyDa 85 14 0 01 Mar 2024
On the Convergence of Differentially-Private Fine-tuning: To Linearly Probe or to Fully Fine-tune? Shuqi Ke Charlie Hou Giulia Fanti Sewoong Oh 83 4 0 29 Feb 2024
Pre-training Differentially Private Models with Limited Public Data Zhiqi Bu Xinwei Zhang Mingyi Hong Sheng Zha George Karypis 121 4 0 28 Feb 2024
Privacy-Preserving Instructions for Aligning Large Language Models Da Yu Peter Kairouz Sewoong Oh Zheng Xu 120 25 0 21 Feb 2024
On Differentially Private Subspace Estimation in a Distribution-Free Setting Eliad Tsfadia 95 2 0 09 Feb 2024
Private Fine-tuning of Large Language Models with Zeroth-order Optimization Xinyu Tang Ashwinee Panda Milad Nasr Saeed Mahloujifar Prateek Mittal 218 26 0 09 Jan 2024
Inference and Interference: The Role of Clipping, Pruning and Loss Landscapes in Differentially Private Stochastic Gradient Descent Lauren Watson Eric Gan Mohan Dantam Baharan Mirzasoleiman Rik Sarkar 54 1 0 12 Nov 2023
PrivLM-Bench: A Multi-level Privacy Evaluation Benchmark for Language Models Haoran Li Dadi Guo Donghao Li Wei Fan Qi Hu Xin Liu Chunkit Chan Duanyi Yao Yuan Yao Yangqiu Song PILM 107 25 0 07 Nov 2023
Initialization Matters: Privacy-Utility Analysis of Overparameterized Neural Networks Jiayuan Ye Zhenyu Zhu Fanghui Liu Reza Shokri Volkan Cevher 91 13 0 31 Oct 2023
FLTrojan: Privacy Leakage Attacks against Federated Language Models Through Selective Weight Tampering Md Rafi Ur Rashid Vishnu Asutosh Dasu Kang Gu Najrin Sultana Shagufta Mehnaz AAML FedML 184 12 0 24 Oct 2023
DPZero: Private Fine-Tuning of Language Models without Backpropagation Liang Zhang Bingcong Li K. K. Thekumparampil Sewoong Oh Niao He 96 15 0 14 Oct 2023
Differentially Private Non-convex Learning for Multi-layer Neural Networks Hanpu Shen Cheng-Long Wang Zihang Xiang Yiming Ying Di Wang 83 8 0 12 Oct 2023
Enabling Differentially Private Federated Learning for Speech Recognition: Benchmarks, Adaptive Optimizers and Gradient Clipping Martin Pelikan Sheikh Shams Azam Vitaly Feldman Jan Honza Silovsky Kunal Talwar Christopher G. Brinton Tatiana Likhomanenko 113 8 0 29 Sep 2023
Geometry of Sensitivity: Twice Sampling and Hybrid Clipping in Differential Privacy with Optimal Gaussian Noise and Application to Deep Learning Hanshen Xiao Jun Wan Srini Devadas 71 8 0 06 Sep 2023
Unlocking Accuracy and Fairness in Differentially Private Image Classification Leonard Berrada Soham De J. Shen Jamie Hayes Robert Stanforth David Stutz Pushmeet Kohli Samuel L. Smith Borja Balle 76 15 0 21 Aug 2023
The importance of feature preprocessing for differentially private linear optimization Ziteng Sun A. Suresh A. Menon 81 3 0 19 Jul 2023
Differentially Private Image Classification by Learning Priors from Random Processes Xinyu Tang Ashwinee Panda Vikash Sehwag Prateek Mittal 91 21 0 08 Jun 2023
PILLAR: How to make semi-private learning more effective Francesco Pinto Yaxian Hu Fanny Yang Amartya Sanyal 105 12 0 06 Jun 2023
Better Private Linear Regression Through Better Private Feature Selection Travis Dick Jennifer Gillenwater Matthew Joseph 65 3 0 01 Jun 2023
Surrogate Model Extension (SME): A Fast and Accurate Weight Update Attack on Federated Learning Junyi Zhu Ruicong Yao Matthew B. Blaschko FedML 89 11 0 31 May 2023
Learning across Data Owners with Joint Differential Privacy Yangsibo Huang Haotian Jiang Daogao Liu Mohammad Mahdian Jieming Mao Vahab Mirrokni FedML 84 0 0 25 May 2023
Differentially Private Synthetic Data via Foundation Model APIs 1: Images Zinan Lin Sivakanth Gopi Janardhan Kulkarni Harsha Nori Sergey Yekhanin 180 44 0 24 May 2023
Selective Pre-training for Private Fine-tuning Da Yu Sivakanth Gopi Janardhan Kulkarni Zinan Lin Saurabh Naik Tomasz Religa Jian Yin Huishuai Zhang 94 19 0 23 May 2023
Shattering the Agent-Environment Interface for Fine-Tuning Inclusive Language Models Wanqiao Xu Shi Dong Dilip Arumugam Benjamin Van Roy 85 8 0 19 May 2023
Auditing and Generating Synthetic Data with Controllable Trust Trade-offs Brian M. Belgodere Pierre Dognin Adam Ivankay Igor Melnyk Youssef Mroueh ... Mattia Rigotti Jerret Ross Yair Schiff Radhika Vedpathak Richard A. Young 143 12 0 21 Apr 2023
Choosing Public Datasets for Private Machine Learning via Gradient Subspace Distance Xin Gu Gautam Kamath Zhiwei Steven Wu 74 14 0 02 Mar 2023
Why Is Public Pretraining Necessary for Private Model Training? Arun Ganesh Mahdi Haghifam Milad Nasr Sewoong Oh Thomas Steinke Om Thakkar Abhradeep Thakurta Lun Wang 70 39 0 19 Feb 2023
On the Efficacy of Differentially Private Few-shot Image Classification Marlon Tobaben Aliaksandra Shysheya J. Bronskill Andrew Paverd Shruti Tople Santiago Zanella Béguelin Richard Turner Antti Honkela 107 12 0 02 Feb 2023
ReSQueing Parallel and Private Stochastic Convex Optimization Y. Carmon A. Jambulapati Yujia Jin Y. Lee Daogao Liu Aaron Sidford Kevin Tian FedML 98 14 0 01 Jan 2023
A New Linear Scaling Rule for Private Adaptive Hyperparameter Optimization Ashwinee Panda Xinyu Tang Saeed Mahloujifar Vikash Sehwag Prateek Mittal 126 12 0 08 Dec 2022
Exploring the Limits of Differentially Private Deep Learning with Group-wise Clipping Jiyan He Xuechen Li Da Yu Huishuai Zhang Janardhan Kulkarni Y. Lee A. Backurs Nenghai Yu Jiang Bian 126 49 0 03 Dec 2022
Differentially Private Image Classification from Features Harsh Mehta Walid Krichene Abhradeep Thakurta Alexey Kurakin Ashok Cutkosky 118 8 0 24 Nov 2022
Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipe Xiang Yue Huseyin A. Inan Xuechen Li Girish Kumar Julia McAnallen Hoda Shajari Huan Sun David Levitan Robert Sim 154 86 0 25 Oct 2022
Differentially Private Diffusion Models Tim Dockhorn Tianshi Cao Arash Vahdat Karsten Kreis DiffM 105 100 0 18 Oct 2022
A Closer Look at the Calibration of Differentially Private Learners Hanlin Zhang Xuechen Li Prithviraj Sen Salim Roukos Tatsunori Hashimoto 84 3 0 15 Oct 2022
On the Convergence and Calibration of Deep Learning with Differential Privacy Zhiqi Bu Hua Wang Zongyu Dai Qi Long 115 31 0 15 Jun 2021