Exploring the Limits of Differentially Private Deep Learning with Group-wise Clipping

3 December 2022

Jiang Bian

Papers citing "Exploring the Limits of Differentially Private Deep Learning with Group-wise Clipping"

47 / 47 papers shown

Title
Privacy Auditing of Large Language Models Ashwinee Panda Xinyu Tang Milad Nasr Christopher A. Choquette-Choo Prateek Mittal PILM 62 5 0 09 Mar 2025
Towards Label-Only Membership Inference Attack against Pre-trained Large Language Models Yu He Boheng Li L. Liu Zhongjie Ba Wei Dong Yiming Li Zhanyue Qin Kui Ren Cheng Chen MIALM 74 0 0 26 Feb 2025
Differentially Private Prototypes for Imbalanced Transfer Learning Dariush Wahdany Matthew Jagielski Adam Dziedzic Franziska Boenisch 88 0 0 17 Feb 2025
Differentially Private Synthetic Data via APIs 3: Using Simulators Instead of Foundation Model Zinan Lin Tadas Baltrusaitis Wenyu Wang Sergey Yekhanin SyDa 91 1 0 08 Feb 2025
Differential Privacy with Higher Utility by Exploiting Coordinate-wise Disparity: Laplace Mechanism Can Beat Gaussian in High Dimensions Gokularam Muthukrishnan Sheetal Kalyani 87 0 0 28 Jan 2025
Balls-and-Bins Sampling for DP-SGD Lynn Chua Badih Ghazi Charlie Harrison Ethan Leeman Pritish Kamath Ravi Kumar Pasin Manurangsi Amer Sinha Chiyuan Zhang 80 4 0 21 Dec 2024
Scalable DP-SGD: Shuffling vs. Poisson Subsampling Lynn Chua Badih Ghazi Pritish Kamath Ravi Kumar Pasin Manurangsi Amer Sinha Chiyuan Zhang 36 5 0 06 Nov 2024
Data-adaptive Differentially Private Prompt Synthesis for In-Context Learning Fengyu Gao Ruida Zhou T. Wang Cong Shen Jing Yang 37 2 0 15 Oct 2024
Weights Shuffling for Improving DPSGD in Transformer-based Models Jungang Yang Zhe Ji Liyao Xiang 40 0 0 22 Jul 2024
LMO-DP: Optimizing the Randomization Mechanism for Differentially Private Fine-Tuning (Large) Language Models Qin Yang Meisam Mohammady Han Wang Ali Payani Ashish Kundu Kai Shu Yan Yan Yuan Hong 28 0 0 29 May 2024
Delving into Differentially Private Transformer Youlong Ding Xueyang Wu Yining Meng Yonggang Luo Hao Wang Weike Pan 39 5 0 28 May 2024
Learnable Privacy Neurons Localization in Language Models Ruizhe Chen Tianxiang Hu Yang Feng Zuo-Qiang Liu 44 12 0 16 May 2024
How Private are DP-SGD Implementations? Lynn Chua Badih Ghazi Pritish Kamath Ravi Kumar Pasin Manurangsi Amer Sinha Chiyuan Zhang 43 12 0 26 Mar 2024
Differentially Private Synthetic Data via Foundation Model APIs 2: Text Chulin Xie Zinan Lin A. Backurs Sivakanth Gopi Da Yu ... Haotian Jiang Huishuai Zhang Yin Tat Lee Bo-wen Li Sergey Yekhanin SyDa 63 34 0 04 Mar 2024
Pre-training Differentially Private Models with Limited Public Data Zhiqi Bu Xinwei Zhang Mingyi Hong Sheng Zha George Karypis 79 3 0 28 Feb 2024
LLM-based Privacy Data Augmentation Guided by Knowledge Distillation with a Distribution Tutor for Medical Text Classification Yiping Song Juhua Zhang Zhiliang Tian Yuxin Yang Minlie Huang Dongsheng Li 39 10 0 26 Feb 2024
Privacy-Preserving Instructions for Aligning Large Language Models Da Yu Peter Kairouz Sewoong Oh Zheng Xu 34 17 0 21 Feb 2024
Differentially Private Zeroth-Order Methods for Scalable Large Language Model Finetuning Zhicheng Liu Jian Lou W. Bao Yihan Hu Baochun Li Zhanyue Qin K. Ren 31 7 0 12 Feb 2024
Differentially Private Training of Mixture of Experts Models Pierre Tholoniat Huseyin A. Inan Janardhan Kulkarni Robert Sim MoE 41 1 0 11 Feb 2024
Private Fine-tuning of Large Language Models with Zeroth-order Optimization Xinyu Tang Ashwinee Panda Milad Nasr Saeed Mahloujifar Prateek Mittal 47 18 0 09 Jan 2024
On the Benefits of Public Representations for Private Transfer Learning under Distribution Shift Pratiksha Thaker Amrith Rajagopal Setlur Zhiwei Steven Wu Virginia Smith 41 2 0 24 Dec 2023
Optimal Unbiased Randomizers for Regression with Label Differential Privacy Ashwinkumar Badanidiyuru Badih Ghazi Pritish Kamath Ravi Kumar Ethan Leeman Pasin Manurangsi A. Varadarajan Chiyuan Zhang 34 4 0 09 Dec 2023
Zero redundancy distributed learning with differential privacy Zhiqi Bu Justin Chiu Ruixuan Liu Sheng Zha George Karypis 45 8 0 20 Nov 2023
On the accuracy and efficiency of group-wise clipping in differentially private optimization Zhiqi Bu Ruixuan Liu Yu-Xiang Wang Sheng Zha George Karypis VLM 35 4 0 30 Oct 2023
DPZero: Private Fine-Tuning of Language Models without Backpropagation Liang Zhang Bingcong Li K. K. Thekumparampil Sewoong Oh Niao He 28 11 0 14 Oct 2023
Correlated Noise Provably Beats Independent Noise for Differentially Private Learning Christopher A. Choquette-Choo Krishnamurthy Dvijotham Krishna Pillutla Arun Ganesh Thomas Steinke Abhradeep Thakurta 27 13 0 10 Oct 2023
Unlocking Accuracy and Fairness in Differentially Private Image Classification Leonard Berrada Soham De J. Shen Jamie Hayes Robert Stanforth David Stutz Pushmeet Kohli Samuel L. Smith Borja Balle 27 13 0 21 Aug 2023
Differentially Private Image Classification by Learning Priors from Random Processes Xinyu Tang Ashwinee Panda Vikash Sehwag Prateek Mittal 23 20 0 08 Jun 2023
TMI! Finetuned Models Leak Private Information from their Pretraining Data John Abascal Stanley Wu Alina Oprea Jonathan R. Ullman 31 16 0 01 Jun 2023
DPFormer: Learning Differentially Private Transformer on Long-Tailed Data Youlong Ding Xueyang Wu Hongya Wang Weike Pan 31 0 0 28 May 2023
DP-SGD Without Clipping: The Lipschitz Neural Network Way Louis Bethune Thomas Massena Thibaut Boissin Yannick Prudent Corentin Friedrich Franck Mamalet A. Bellet M. Serrurier David Vigouroux 34 9 0 25 May 2023
Training Data Extraction From Pre-trained Language Models: A Survey Shotaro Ishihara 26 46 0 25 May 2023
Differentially Private Synthetic Data via Foundation Model APIs 1: Images Zinan Lin Sivakanth Gopi Janardhan Kulkarni Harsha Nori Sergey Yekhanin 41 36 0 24 May 2023
Selective Pre-training for Private Fine-tuning Da Yu Sivakanth Gopi Janardhan Kulkarni Zinan Lin Saurabh Naik Tomasz Religa Jian Yin Huishuai Zhang 35 19 0 23 May 2023
Privacy-Preserving In-Context Learning for Large Language Models Tong Wu Ashwinee Panda Jiachen T. Wang Prateek Mittal 51 29 0 02 May 2023
Why Is Public Pretraining Necessary for Private Model Training? Arun Ganesh Mahdi Haghifam Milad Nasr Sewoong Oh Thomas Steinke Om Thakkar Abhradeep Thakurta Lun Wang 26 36 0 19 Feb 2023
Private Ad Modeling with DP-SGD Carson E. Denison Badih Ghazi Pritish Kamath Ravi Kumar Pasin Manurangsi Krishnagiri Narra Amer Sinha A. Varadarajan Chiyuan Zhang 32 14 0 21 Nov 2022
Automatic Clipping: Differentially Private Deep Learning Made Easier and Stronger Zhiqi Bu Yu-Xiang Wang Sheng Zha George Karypis 27 69 0 14 Jun 2022
Scalable and Efficient Training of Large Convolutional Neural Networks with Differential Privacy Zhiqi Bu J. Mao Shiyun Xu 139 47 0 21 May 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 319 11,953 0 04 Mar 2022
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 213 1,657 0 15 Oct 2021
Differentially Private Fine-tuning of Language Models Da Yu Saurabh Naik A. Backurs Sivakanth Gopi Huseyin A. Inan ... Y. Lee Andre Manoel Lukas Wutschitz Sergey Yekhanin Huishuai Zhang 134 347 0 13 Oct 2021
Opacus: User-Friendly Differential Privacy Library in PyTorch Ashkan Yousefpour I. Shilov Alexandre Sablayrolles Davide Testuggine Karthik Prasad ... Sayan Gosh Akash Bharadwaj Jessica Zhao Graham Cormode Ilya Mironov VLM 168 350 0 25 Sep 2021
Do Not Let Privacy Overbill Utility: Gradient Embedding Perturbation for Private Learning Da Yu Huishuai Zhang Wei Chen Tie-Yan Liu FedML SILM 94 110 0 25 Feb 2021
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown D. Song Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 290 1,815 0 14 Dec 2020
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,959 0 20 Apr 2018
Efficient Per-Example Gradient Computations Ian Goodfellow 186 74 0 07 Oct 2015