v1v2v3 (latest)

Private Fine-tuning of Large Language Models with Zeroth-order Optimization

9 January 2024

Papers citing "Private Fine-tuning of Large Language Models with Zeroth-order Optimization"

32 / 82 papers shown

Title
Optimal Accounting of Differential Privacy via Characteristic Function Yuqing Zhu Jinshuo Dong Yu Wang 55 102 0 16 Jun 2021
On the Convergence and Calibration of Deep Learning with Differential Privacy Zhiqi Bu Hua Wang Zongyu Dai Qi Long 72 31 0 15 Jun 2021
Numerical Composition of Differential Privacy Sivakanth Gopi Y. Lee Lukas Wutschitz 61 183 0 05 Jun 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 579 4,077 0 18 Apr 2021
Do Not Let Privacy Overbill Utility: Gradient Embedding Perturbation for Private Learning Da Yu Huishuai Zhang Wei Chen Tie-Yan Liu FedML SILM 132 115 0 25 Feb 2021
Computing Differential Privacy Guarantees for Heterogeneous Compositions Using FFT A. Koskela Antti Honkela 44 20 0 24 Feb 2021
Prefix-Tuning: Optimizing Continuous Prompts for Generation Xiang Lisa Li Percy Liang 248 4,298 0 01 Jan 2021
Making Pre-trained Language Models Better Few-shot Learners Tianyu Gao Adam Fisch Danqi Chen 402 1,971 0 31 Dec 2020
Tight Differential Privacy for Discrete-Valued Mechanisms and for the Subsampled Gaussian Mechanism Using FFT A. Koskela Hibiki Ito Lukas Prediger Antti Honkela 48 59 0 12 Jun 2020
Stability of Stochastic Gradient Descent on Nonsmooth Convex Losses Raef Bassily Vitaly Feldman Cristóbal Guzmán Kunal Talwar MLT 54 198 0 12 Jun 2020
A Primer on Zeroth-Order Optimization in Signal Processing and Machine Learning Sijia Liu Pin-Yu Chen B. Kailkhura Gaoyuan Zhang A. Hero III P. Varshney 72 235 0 11 Jun 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 838 42,332 0 28 May 2020
ZeRO: Memory Optimizations Toward Training Trillion Parameter Models Samyam Rajbhandari Jeff Rasley Olatunji Ruwase Yuxiong He ALM AI4CE 82 902 0 04 Oct 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 674 24,528 0 26 Jul 2019
SoK: Differential Privacies Damien Desfontaines Balázs Pejó 66 124 0 04 Jun 2019
Differentially Private Learning with Adaptive Clipping Galen Andrew Om Thakkar H. B. McMahan Swaroop Ramaswamy FedML 84 341 0 09 May 2019
DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs Dheeru Dua Yizhong Wang Pradeep Dasigi Gabriel Stanovsky Sameer Singh Matt Gardner AIMat 103 963 0 01 Mar 2019
DP-ADMM: ADMM-based Distributed Learning with Differential Privacy Zonghao Huang Rui Hu Yuanxiong Guo Eric Chan-Tin Yanmin Gong FedML 66 197 0 30 Aug 2018
Privacy Amplification by Subsampling: Tight Analyses via Couplings and Divergences Borja Balle Gilles Barthe Marco Gaboardi 84 392 0 04 Jul 2018
Zeroth-Order Stochastic Variance Reduction for Nonconvex Optimization Sijia Liu B. Kailkhura Pin-Yu Chen Pai-Shun Ting Shiyu Chang Lisa Amini 97 185 0 25 May 2018
Improving the Gaussian Mechanism for Differential Privacy: Analytical Calibration and Optimal Denoising Borja Balle Yu Wang MLT 80 408 0 16 May 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 1.1K 7,182 0 20 Apr 2018
Renyi Differential Privacy Ilya Mironov 77 1,263 0 24 Feb 2017
Membership Inference Attacks against Machine Learning Models Reza Shokri M. Stronati Congzheng Song Vitaly Shmatikov SLR MIALM MIACV 269 4,152 0 18 Oct 2016
Deep Learning with Differential Privacy Martín Abadi Andy Chu Ian Goodfellow H. B. McMahan Ilya Mironov Kunal Talwar Li Zhang FedML SyDa 216 6,155 0 01 Jul 2016
SQuAD: 100,000+ Questions for Machine Comprehension of Text Pranav Rajpurkar Jian Zhang Konstantin Lopyrev Percy Liang RALM 312 8,160 0 16 Jun 2016
Concentrated Differential Privacy Cynthia Dwork G. Rothblum 70 453 0 06 Mar 2016
Proving Differential Privacy via Probabilistic Couplings Gilles Barthe Marco Gaboardi B. Grégoire Justin Hsu Pierre-Yves Strub 95 103 0 19 Jan 2016
Optimal rates for zero-order convex optimization: the power of two function evaluations John C. Duchi Michael I. Jordan Martin J. Wainwright Andre Wibisono 79 489 0 07 Dec 2013
The Composition Theorem for Differential Privacy Peter Kairouz Sewoong Oh Pramod Viswanath 123 682 0 04 Nov 2013
Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programming Saeed Ghadimi Guanghui Lan ODL 122 1,555 0 22 Sep 2013
On the Complexity of Bandit and Derivative-Free Stochastic Convex Optimization Ohad Shamir 417 193 0 11 Sep 2012