v1v2 (latest)

Can Public Large Language Models Help Private Cross-device Federated Learning?

20 May 2023

Papers citing "Can Public Large Language Models Help Private Cross-device Federated Learning?"

21 / 21 papers shown

Title
A Hassle-free Algorithm for Private Learning in Practice: Don't Use Tree Aggregation, Use BLTs H. B. McMahan Zheng Xu Yanxiang Zhang FedML 106 8 0 16 Aug 2024
Federated Learning of Gboard Language Models with Differential Privacy Zheng Xu Yanxiang Zhang Galen Andrew Christopher A. Choquette-Choo Peter Kairouz H. B. McMahan Jesse Rosenstock Yuanbo Zhang FedML 88 81 0 29 May 2023
Where to Begin? On the Impact of Pre-Training and Initialization in Federated Learning John Nguyen Jianyu Wang Kshitiz Malik Maziar Sanjabi Michael G. Rabbat FedML AI4CE 87 73 0 14 Oct 2022
Motley: Benchmarking Heterogeneity and Personalization in Federated Learning Shan-shan Wu Tian Li Zachary B. Charles Yu Xiao Ziyu Liu Zheng Xu Virginia Smith FedML 101 45 0 18 Jun 2022
Private Adaptive Optimization with Side Information Tian Li Manzil Zaheer Sashank J. Reddi Virginia Smith 78 36 0 12 Feb 2022
LaMDA: Language Models for Dialog Applications R. Thoppilan Daniel De Freitas Jamie Hall Noam M. Shazeer Apoorv Kulshreshtha ... Blaise Aguera-Arcas Claire Cui M. Croak Ed H. Chi Quoc Le ALM 142 1,601 0 20 Jan 2022
Public Data-Assisted Mirror Descent for Private Model Training Ehsan Amid Arun Ganesh Rajiv Mathews Swaroop Indra Ramaswamy Shuang Song Thomas Steinke Vinith Suriyakumar Om Thakkar Abhradeep Thakurta 86 51 0 01 Dec 2021
Differentially Private Fine-tuning of Language Models Da Yu Saurabh Naik A. Backurs Sivakanth Gopi Huseyin A. Inan ... Y. Lee Andre Manoel Lukas Wutschitz Sergey Yekhanin Huishuai Zhang 240 371 0 13 Oct 2021
A Field Guide to Federated Optimization Jianyu Wang Zachary B. Charles Zheng Xu Gauri Joshi H. B. McMahan ... Mi Zhang Tong Zhang Chunxiang Zheng Chen Zhu Wennan Zhu FedML 273 421 0 14 Jul 2021
Training Production Language Models without Memorizing User Data Swaroop Indra Ramaswamy Om Thakkar Rajiv Mathews Galen Andrew H. B. McMahan Franccoise Beaufays FedML 67 92 0 21 Sep 2020
Differentially Private Language Models Benefit from Public Pre-training Gavin Kerrigan Dylan Slack Jens Tuyls 79 57 0 13 Sep 2020
MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices Zhiqing Sun Hongkun Yu Xiaodan Song Renjie Liu Yiming Yang Denny Zhou MQ 115 817 0 06 Apr 2020
Adaptive Federated Optimization Sashank J. Reddi Zachary B. Charles Manzil Zaheer Zachary Garrett Keith Rush Jakub Konecný Sanjiv Kumar H. B. McMahan FedML 183 1,452 0 29 Feb 2020
MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers Wenhui Wang Furu Wei Li Dong Hangbo Bao Nan Yang Ming Zhou VLM 176 1,282 0 25 Feb 2020
Advances and Open Problems in Federated Learning Peter Kairouz H. B. McMahan Brendan Avent A. Bellet M. Bennis ... Zheng Xu Qiang Yang Felix X. Yu Han Yu Sen Zhao FedML AI4CE 275 6,294 0 10 Dec 2019
TinyBERT: Distilling BERT for Natural Language Understanding Xiaoqi Jiao Yichun Yin Lifeng Shang Xin Jiang Xiao Chen Linlin Li F. Wang Qun Liu VLM 113 1,872 0 23 Sep 2019
SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing Taku Kudo John Richardson 206 3,531 0 19 Aug 2018
Automated Scalable Bayesian Inference via Hilbert Coresets Trevor Campbell Tamara Broderick 78 127 0 13 Oct 2017
Deep Learning with Differential Privacy Martín Abadi Andy Chu Ian Goodfellow H. B. McMahan Ilya Mironov Kunal Talwar Li Zhang FedML SyDa 216 6,172 0 01 Jul 2016
Communication-Efficient Learning of Deep Networks from Decentralized Data H. B. McMahan Eider Moore Daniel Ramage S. Hampson Blaise Agüera y Arcas FedML 408 17,615 0 17 Feb 2016
Neural Machine Translation of Rare Words with Subword Units Rico Sennrich Barry Haddow Alexandra Birch 235 7,760 0 31 Aug 2015