Large-Scale Differentially Private BERT

3 August 2021

Papers citing "Large-Scale Differentially Private BERT"

50 / 107 papers shown

Title
Differentially Private Synthetic Data via Foundation Model APIs 1: Images Zi-Han Lin Sivakanth Gopi Janardhan Kulkarni Harsha Nori Sergey Yekhanin 39 36 0 24 May 2023
Flocks of Stochastic Parrots: Differentially Private Prompt Learning for Large Language Models Haonan Duan Adam Dziedzic Nicolas Papernot Franziska Boenisch AAML 19 62 0 24 May 2023
Selective Pre-training for Private Fine-tuning Da Yu Sivakanth Gopi Janardhan Kulkarni Zi-Han Lin Saurabh Naik Tomasz Religa Jian Yin Huishuai Zhang 32 19 0 23 May 2023
Quantifying Association Capabilities of Large Language Models and Its Implications on Privacy Leakage Hanyin Shao Jie Huang Shen Zheng Kevin Chen-Chuan Chang PILM 22 24 0 22 May 2023
Privacy-Preserving Prompt Tuning for Large Language Model Services Yansong Li Zhixing Tan Yang Liu SILM VLM 45 63 0 10 May 2023
Mitigating Approximate Memorization in Language Models via Dissimilarity Learned Policy Aly M. Kassem 26 2 0 02 May 2023
Emergent and Predictable Memorization in Large Language Models Stella Biderman USVSN Sai Prashanth Lintang Sutawika Hailey Schoelkopf Quentin G. Anthony Shivanshu Purohit Edward Raf 24 116 0 21 Apr 2023
The MiniPile Challenge for Data-Efficient Language Models Jean Kaddour MoE ALM 24 40 0 17 Apr 2023
Practical Differentially Private and Byzantine-resilient Federated Learning Zihang Xiang Tianhao Wang Wanyu Lin Di Wang FedML 31 21 0 15 Apr 2023
Foundation Models and Fair Use Peter Henderson Xuechen Li Dan Jurafsky Tatsunori Hashimoto Mark A. Lemley Percy Liang 22 119 0 28 Mar 2023
How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy Natalia Ponomareva Hussein Hazimeh Alexey Kurakin Zheng Xu Carson E. Denison H. B. McMahan Sergei Vassilvitskii Steve Chien Abhradeep Thakurta 94 167 0 01 Mar 2023
One-shot Empirical Privacy Estimation for Federated Learning Galen Andrew Peter Kairouz Sewoong Oh Alina Oprea H. B. McMahan Vinith M. Suriyakumar FedML 21 32 0 06 Feb 2023
Private GANs, Revisited Alex Bie Gautam Kamath Guojun Zhang 8 14 0 06 Feb 2023
Differentially Private Natural Language Models: Recent Advances and Future Directions Lijie Hu Ivan Habernal Lei Shen Di Wang AAML 15 18 0 22 Jan 2023
Training Differentially Private Graph Neural Networks with Random Walk Sampling Morgane Ayle Jan Schuchardt Lukas Gosch Daniel Zügner Stephan Günnemann FedML 21 6 0 02 Jan 2023
Tensions Between the Proxies of Human Values in AI Teresa Datta D. Nissani Max Cembalest Akash Khanna Haley Massa John P. Dickerson 31 2 0 14 Dec 2022
Position: Considerations for Differentially Private Learning with Large-Scale Public Pretraining Florian Tramèr Gautam Kamath Nicholas Carlini SILM 38 67 0 13 Dec 2022
Exploring the Limits of Differentially Private Deep Learning with Group-wise Clipping Jiyan He Xuechen Li Da Yu Huishuai Zhang Janardhan Kulkarni Y. Lee A. Backurs Nenghai Yu Jiang Bian 19 46 0 03 Dec 2022
Differentially Private Image Classification from Features Harsh Mehta Walid Krichene Abhradeep Thakurta Alexey Kurakin Ashok Cutkosky 46 7 0 24 Nov 2022
Private Ad Modeling with DP-SGD Carson E. Denison Badih Ghazi Pritish Kamath Ravi Kumar Pasin Manurangsi Krishnagiri Narra Amer Sinha A. Varadarajan Chiyuan Zhang 27 14 0 21 Nov 2022
Learning to Generate Image Embeddings with User-level Differential Privacy Zheng Xu Maxwell D. Collins Yuxiao Wang Liviu Panait Sewoong Oh S. Augenstein Ting Liu Florian Schroff H. B. McMahan FedML 30 29 0 20 Nov 2022
Directional Privacy for Deep Learning Pedro Faustini Natasha Fernandes Shakila Mahjabin Tonni Annabelle McIver Mark Dras 14 1 0 09 Nov 2022
Privacy-Preserving Models for Legal Natural Language Processing Ying Yin Ivan Habernal PILM AILaw 4 8 0 05 Nov 2022
Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacy Daphne Ippolito Florian Tramèr Milad Nasr Chiyuan Zhang Matthew Jagielski Katherine Lee Christopher A. Choquette-Choo Nicholas Carlini PILM MU 23 58 0 31 Oct 2022
Differentially Private Language Models for Secure Data Sharing Justus Mattern Zhijing Jin Benjamin Weggenmann Bernhard Schoelkopf Mrinmaya Sachan SyDa 19 47 0 25 Oct 2022
Differentially Private Diffusion Models Tim Dockhorn Tianshi Cao Arash Vahdat Karsten Kreis DiffM 19 91 0 18 Oct 2022
TAN Without a Burn: Scaling Laws of DP-SGD Tom Sander Pierre Stock Alexandre Sablayrolles FedML 19 42 0 07 Oct 2022
Knowledge Unlearning for Mitigating Privacy Risks in Language Models Joel Jang Dongkeun Yoon Sohee Yang Sungmin Cha Moontae Lee Lajanugen Logeswaran Minjoon Seo KELM PILM MU 147 190 0 04 Oct 2022
On the Impossible Safety of Large AI Models El-Mahdi El-Mhamdi Sadegh Farhadkhani R. Guerraoui Nirupam Gupta L. Hoang Rafael Pinot Sébastien Rouault John Stephan 30 31 0 30 Sep 2022
DiVa: An Accelerator for Differentially Private Machine Learning Beom-Joo Park Ranggi Hwang Dongho Yoon Yoonhyuk Choi Minsoo Rhu 11 8 0 26 Aug 2022
Training Large-Vocabulary Neural Language Models by Private Federated Learning for Resource-Constrained Devices Mingbin Xu Congzheng Song Ye Tian Neha Agrawal Filip Granqvist ... Shiyi Han Yaqiao Deng Leo Liu Anmol Walia Alex Jin FedML 13 22 0 18 Jul 2022
Combing for Credentials: Active Pattern Extraction from Smart Reply Bargav Jayaraman Esha Ghosh Melissa Chase Sambuddha Roy Wei Dai David E. Evans SILM 20 8 0 14 Jul 2022
A Customized Text Sanitization Mechanism with Differential Privacy Hui Chen Fengran Mo Yanhao Wang Cen Chen J. Nie Chengyu Wang Jamie Cui 11 34 0 04 Jul 2022
Individual Privacy Accounting for Differentially Private Stochastic Gradient Descent Da Yu Gautam Kamath Janardhan Kulkarni Tie-Yan Liu Jian Yin Huishuai Zhang 11 17 0 06 Jun 2022
Differentially Private Model Compression Fatemehsadat Mireshghallah A. Backurs Huseyin A. Inan Lukas Wutschitz Janardhan Kulkarni SyDa 16 13 0 03 Jun 2022
Are Large Pre-Trained Language Models Leaking Your Personal Information? Jie Huang Hanyin Shao Kevin Chen-Chuan Chang PILM 17 177 0 25 May 2022
Large Scale Transfer Learning for Differentially Private Image Classification Harsh Mehta Abhradeep Thakurta Alexey Kurakin Ashok Cutkosky 9 39 0 06 May 2022
Provably Confidential Language Modelling Xuandong Zhao Lei Li Yu-Xiang Wang MU 14 15 0 04 May 2022
Unlocking High-Accuracy Differentially Private Image Classification through Scale Soham De Leonard Berrada Jamie Hayes Samuel L. Smith Borja Balle 28 217 0 28 Apr 2022
Just Fine-tune Twice: Selective Differential Privacy for Large Language Models Weiyan Shi Ryan Shea Si-An Chen Chiyuan Zhang R. Jia Zhou Yu AAML 26 38 0 15 Apr 2022
Quantifying Memorization Across Neural Language Models Nicholas Carlini Daphne Ippolito Matthew Jagielski Katherine Lee Florian Tramèr Chiyuan Zhang PILM 22 579 0 15 Feb 2022
What Does it Mean for a Language Model to Preserve Privacy? Hannah Brown Katherine Lee Fatemehsadat Mireshghallah Reza Shokri Florian Tramèr PILM 29 232 0 11 Feb 2022
Step-size Adaptation Using Exponentiated Gradient Updates Ehsan Amid Rohan Anil Christopher Fifty Manfred K. Warmuth 35 9 0 31 Jan 2022
Toward Training at ImageNet Scale with Differential Privacy Alexey Kurakin Shuang Song Steve Chien Roxana Geambasu Andreas Terzis Abhradeep Thakurta 30 99 0 28 Jan 2022
DP-FP: Differentially Private Forward Propagation for Large Models Jian Du Haitao Mi 19 5 0 29 Dec 2021
Node-Level Differentially Private Graph Neural Networks Ameya Daigavane Gagan Madan Aditya Sinha Abhradeep Thakurta Gaurav Aggarwal Prateek Jain 28 54 0 23 Nov 2021
Differentially Private Fine-tuning of Language Models Da Yu Saurabh Naik A. Backurs Sivakanth Gopi Huseyin A. Inan ... Y. Lee Andre Manoel Lukas Wutschitz Sergey Yekhanin Huishuai Zhang 134 346 0 13 Oct 2021
Large Language Models Can Be Strong Differentially Private Learners Xuechen Li Florian Tramèr Percy Liang Tatsunori Hashimoto 22 365 0 12 Oct 2021
NanoBatch Privacy: Enabling fast Differentially Private learning on the IPU Edward H. Lee M. M. Krell Alexander Tsyplikhin Victoria Rege E. Colak Kristen W. Yeom FedML 16 0 0 24 Sep 2021
Trustworthy AI: A Computational Perspective Haochen Liu Yiqi Wang Wenqi Fan Xiaorui Liu Yaxin Li Shaili Jain Yunhao Liu Anil K. Jain Jiliang Tang FaML 98 196 0 12 Jul 2021