Synthesizing Privacy-Preserving Text Data via Finetuning without Finetuning Billion-Scale LLMs

16 March 2025

Papers citing "Synthesizing Privacy-Preserving Text Data via Finetuning without Finetuning Billion-Scale LLMs"

39 / 39 papers shown

Title
Synthesizing and Adapting Error Correction Data for Mobile Large Language Model Applications Yanxiang Zhang Zheng Xu Shanshan Wu Yuanbo Zhang Daniel Ramage KELM 22 0 0 24 May 2025
Private Text Generation by Seeding Large Language Model Prompts Supriya Nagesh Justin Y. Chen Nina Mishra Tal Wagner SyDa SILM 94 2 0 20 Feb 2025
Federated Learning in Practice: Reflections and Projections Katharine Daly Hubert Eichner Peter Kairouz H. B. McMahan Daniel Ramage Zheng Xu FedML 94 10 0 11 Oct 2024
KnowledgeSG: Privacy-Preserving Synthetic Text Generation with Knowledge Distillation from Server Wenhao Wang Xiaoyu Liang Rui Ye Jingyi Chai Siheng Chen Yanfeng Wang SyDa 77 6 0 08 Oct 2024
Gemma 2: Improving Open Language Models at a Practical Size Gemma Team Gemma Team Morgane Riviere Shreya Pathak Pier Giuseppe Sessa Cassidy Hardin ... Noah Fiedel Armand Joulin Kathleen Kenealy Robert Dadashi Alek Andreev VLM MoE OSLM 143 916 0 31 Jul 2024
Private prediction for large-scale synthetic text generation Kareem Amin Alex Bie Weiwei Kong Alexey Kurakin Natalia Ponomareva Umar Syed Andreas Terzis Sergei Vassilvitskii SyDa SILM 134 6 0 16 Jul 2024
PrE-Text: Training Language Models on Private Federated Data in the Age of LLMs Charlie Hou Akshat Shrivastava Hongyuan Zhan Rylan Conway Trang Le Adithya Sagar Giulia Fanti Daniel Lazar 97 15 0 05 Jun 2024
Prompt Public Large Language Models to Synthesize Data for Private On-device Applications Shanshan Wu Zheng Xu Yanxiang Zhang Yuanbo Zhang Daniel Ramage SyDa 69 12 0 05 Apr 2024
Differentially Private Synthetic Data via Foundation Model APIs 2: Text Chulin Xie Zinan Lin A. Backurs Sivakanth Gopi Da Yu ... Haotian Jiang Huishuai Zhang Yin Tat Lee Yue Liu Sergey Yekhanin SyDa 108 44 0 04 Mar 2024
Privacy-Preserving Instructions for Aligning Large Language Models Da Yu Peter Kairouz Sewoong Oh Zheng Xu 93 25 0 21 Feb 2024
Infini-gram: Scaling Unbounded n-gram Language Models to a Trillion Tokens Jiacheng Liu Sewon Min Luke Zettlemoyer Yejin Choi Hannaneh Hajishirzi 128 60 0 30 Jan 2024
Scalable Extraction of Training Data from (Production) Language Models Milad Nasr Nicholas Carlini Jonathan Hayase Matthew Jagielski A. Feder Cooper Daphne Ippolito Christopher A. Choquette-Choo Eric Wallace Florian Tramèr Katherine Lee SILM 73 356 0 28 Nov 2023
Locally Differentially Private Document Generation Using Zero Shot Prompting Saiteja Utpala Sara Hooker Pin-Yu Chen 45 39 0 24 Oct 2023
Privacy-Preserving In-Context Learning with Differentially Private Few-Shot Generation Xinyu Tang Richard Shin Huseyin A. Inan Andre Manoel Fatemehsadat Mireshghallah Zinan Lin Sivakanth Gopi Janardhan Kulkarni Robert Sim 118 60 0 21 Sep 2023
Harnessing large-language models to generate private synthetic text Alexey Kurakin Natalia Ponomareva Umar Syed Liam MacDermed Andreas Terzis SILM SyDa 76 42 0 02 Jun 2023
Federated Learning of Gboard Language Models with Differential Privacy Zheng Xu Yanxiang Zhang Galen Andrew Christopher A. Choquette-Choo Peter Kairouz H. B. McMahan Jesse Rosenstock Yuanbo Zhang FedML 108 81 0 29 May 2023
Flocks of Stochastic Parrots: Differentially Private Prompt Learning for Large Language Models Haonan Duan Adam Dziedzic Nicolas Papernot Franziska Boenisch AAML 70 67 0 24 May 2023
Differentially Private Synthetic Data via Foundation Model APIs 1: Images Zinan Lin Sivakanth Gopi Janardhan Kulkarni Harsha Nori Sergey Yekhanin 139 44 0 24 May 2023
Synthetic Query Generation for Privacy-Preserving Deep Retrieval Systems using Differentially Private Language Models Aldo G. Carranza Rezsa Farahani Natalia Ponomareva Alexey Kurakin Matthew Jagielski Milad Nasr SyDa 51 7 0 10 May 2023
TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings N. Jouppi George Kurian Sheng Li Peter C. Ma R. Nagarajan ... Brian Towles C. Young Xiaoping Zhou Zongwei Zhou David A. Patterson BDL VLM 149 368 0 04 Apr 2023
How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy Natalia Ponomareva Hussein Hazimeh Alexey Kurakin Zheng Xu Carson E. Denison H. B. McMahan Sergei Vassilvitskii Steve Chien Abhradeep Thakurta 152 181 0 01 Mar 2023
LLaMA: Open and Efficient Foundation Language Models Hugo Touvron Thibaut Lavril Gautier Izacard Xavier Martinet Marie-Anne Lachaux ... Faisal Azhar Aurelien Rodriguez Armand Joulin Edouard Grave Guillaume Lample ALM PILM 1.5K 13,490 0 27 Feb 2023
Extracting Training Data from Diffusion Models Nicholas Carlini Jamie Hayes Milad Nasr Matthew Jagielski Vikash Sehwag Florian Tramèr Borja Balle Daphne Ippolito Eric Wallace DiffM 142 618 0 30 Jan 2023
Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipe Xiang Yue Huseyin A. Inan Xuechen Li Girish Kumar Julia McAnallen Hoda Shajari Huan Sun David Levitan Robert Sim 142 86 0 25 Oct 2022
Differentially Private Language Models for Secure Data Sharing Justus Mattern Zhijing Jin Benjamin Weggenmann Bernhard Schoelkopf Mrinmaya Sachan SyDa 86 51 0 25 Oct 2022
The Limits of Word Level Differential Privacy Justus Mattern Benjamin Weggenmann Florian Kerschbaum 55 50 0 02 May 2022
BERTopic: Neural topic modeling with a class-based TF-IDF procedure M. Grootendorst 163 1,498 0 11 Mar 2022
Differentially Private Fine-tuning of Language Models Da Yu Saurabh Naik A. Backurs Sivakanth Gopi Huseyin A. Inan ... Y. Lee Andre Manoel Lukas Wutschitz Sergey Yekhanin Huishuai Zhang 242 372 0 13 Oct 2021
TEM: High Utility Metric Differential Privacy on Text Ricardo Silva Carvalho Theodore Vasiloudis Oluwaseyi Feyisetan 68 36 0 16 Jul 2021
Beyond Goldfish Memory: Long-Term Open-Domain Conversation Jing Xu Arthur Szlam Jason Weston RALM 62 256 0 15 Jul 2021
MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers Krishna Pillutla Swabha Swayamdipta Rowan Zellers John Thickstun Sean Welleck Yejin Choi Zaïd Harchaoui 131 363 0 02 Feb 2021
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown Basel Alomair Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 520 1,956 0 14 Dec 2020
Advances and Open Problems in Federated Learning Peter Kairouz H. B. McMahan Brendan Avent A. Bellet M. Bennis ... Zheng Xu Qiang Yang Felix X. Yu Han Yu Sen Zhao FedML AI4CE 279 6,307 0 10 Dec 2019
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension M. Lewis Yinhan Liu Naman Goyal Marjan Ghazvininejad Abdel-rahman Mohamed Omer Levy Veselin Stoyanov Luke Zettlemoyer AIMat VLM 266 10,880 0 29 Oct 2019
Privacy- and Utility-Preserving Textual Analysis via Calibrated Multivariate Perturbations Oluwaseyi Feyisetan Borja Balle Thomas Drake Tom Diethe 62 157 0 20 Oct 2019
Federated Learning for Mobile Keyboard Prediction Andrew Straiton Hard Kanishka Rao Zhifeng Lin Swaroop Indra Ramaswamy Youjie Li S. Augenstein Alex Schwing M. Annavaram A. Avestimehr FedML 138 1,549 0 08 Nov 2018
Deep Learning with Differential Privacy Martín Abadi Andy Chu Ian Goodfellow H. B. McMahan Ilya Mironov Kunal Talwar Li Zhang FedML SyDa 220 6,172 0 01 Jul 2016
Communication-Efficient Learning of Deep Networks from Decentralized Data H. B. McMahan Eider Moore Daniel Ramage S. Hampson Blaise Agüera y Arcas FedML 414 17,615 0 17 Feb 2016
Differentially Private Empirical Risk Minimization: Efficient Algorithms and Tight Error Bounds Raef Bassily Adam D. Smith Abhradeep Thakurta FedML 150 371 0 27 May 2014