FedKD: Communication Efficient Federated Learning via Knowledge Distillation

30 August 2021

Xing Xie

Papers citing "FedKD: Communication Efficient Federated Learning via Knowledge Distillation"

42 / 42 papers shown

Title
LLM-QFL: Distilling Large Language Model for Quantum Federated Learning Dev Gurung Shiva Raj Pokhrel FedML 158 0 0 24 May 2025
Soft-Label Caching and Sharpening for Communication-Efficient Federated Distillation Kitsuya Azuma Takayuki Nishio Yuichi Kitagawa Wakako Nakano Takahito Tanimura FedML 144 0 0 28 Apr 2025
FedSA: A Unified Representation Learning via Semantic Anchors for Prototype-based Federated Learning Yanbing Zhou Xiangmou Qu Chenlong You Jiyang Zhou Jingyue Tang Xin Zheng Chunmao Cai Yingbo Wu FedML 100 3 0 09 Jan 2025
Explainable Semantic Federated Learning Enabled Industrial Edge Network for Fire Surveillance Li Dong Yubo Peng Feibo Jiang Kezhi Wang Kun Yang 70 0 0 31 Dec 2024
Overcoming Data and Model Heterogeneities in Decentralized Federated Learning via Synthetic Anchors Chun-Yin Huang Kartik Srinivas Xin Zhang Xiaoxiao Li DD 102 6 0 19 May 2024
Stochastic Controlled Averaging for Federated Learning with Communication Compression Xinmeng Huang Ping Li Xiaoyun Li 70 205 0 16 Aug 2023
When Foundation Model Meets Federated Learning: Motivations, Challenges, and Future Directions Weiming Zhuang Chen Chen Lingjuan Lyu Chong Chen Yaochu Jin Lingjuan Lyu AIFin AI4CE 147 95 0 27 Jun 2023
FedICT: Federated Multi-task Distillation for Multi-access Edge Computing Zhiyuan Wu Sheng Sun Yuwei Wang Min Liu Quyang Pan Xue Jiang Bo Gao 92 33 0 01 Jan 2023
Fast Federated Learning by Balancing Communication Trade-Offs Milad Khademi Nori Sangseok Yun Il-Min Kim FedML 51 55 0 23 May 2021
Pufferfish: Communication-efficient Models At No Extra Cost Hongyi Wang Saurabh Agarwal Dimitris Papailiopoulos 55 59 0 05 Mar 2021
Federated Knowledge Distillation Hyowoon Seo Jihong Park Seungeun Oh M. Bennis Seong-Lyun Kim FedML 61 92 0 04 Nov 2020
Federated Model Distillation with Noise-Free Differential Privacy Lichao Sun Lingjuan Lyu FedML 61 107 0 11 Sep 2020
FetchSGD: Communication-Efficient Federated Learning with Sketching D. Rothchild Ashwinee Panda Enayat Ullah Nikita Ivkin Ion Stoica Vladimir Braverman Joseph E. Gonzalez Raman Arora FedML 60 367 0 15 Jul 2020
Ensemble Distillation for Robust Model Fusion in Federated Learning Tao R. Lin Lingjing Kong Sebastian U. Stich Martin Jaggi FedML 97 1,037 0 12 Jun 2020
Pre-trained Models for Natural Language Processing: A Survey Xipeng Qiu Tianxiang Sun Yige Xu Yunfan Shao Ning Dai Xuanjing Huang LM&MA VLM 344 1,480 0 18 Mar 2020
UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training Hangbo Bao Li Dong Furu Wei Wenhui Wang Nan Yang ... Yu Wang Songhao Piao Jianfeng Gao Ming Zhou H. Hon AI4CE 84 393 0 28 Feb 2020
MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers Wenhui Wang Furu Wei Li Dong Hangbo Bao Nan Yang Ming Zhou VLM 120 1,260 0 25 Feb 2020
Advances and Open Problems in Federated Learning Peter Kairouz H. B. McMahan Brendan Avent A. Bellet M. Bennis ... Zheng Xu Qiang Yang Felix X. Yu Han Yu Sen Zhao FedML AI4CE 183 6,229 0 10 Dec 2019
FedMD: Heterogenous Federated Learning via Model Distillation Daliang Li Junpu Wang FedML 88 851 0 08 Oct 2019
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter Victor Sanh Lysandre Debut Julien Chaumond Thomas Wolf 190 7,465 0 02 Oct 2019
FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization Amirhossein Reisizadeh Aryan Mokhtari Hamed Hassani Ali Jadbabaie Ramtin Pedarsani FedML 233 772 0 28 Sep 2019
TinyBERT: Distilling BERT for Natural Language Understanding Xiaoqi Jiao Yichun Yin Lifeng Shang Xin Jiang Xiao Chen Linlin Li F. Wang Qun Liu VLM 92 1,855 0 23 Sep 2019
Patient Knowledge Distillation for BERT Model Compression S. Sun Yu Cheng Zhe Gan Jingjing Liu 118 836 0 25 Aug 2019
Federated Learning: Challenges, Methods, and Future Directions Tian Li Anit Kumar Sahu Ameet Talwalkar Virginia Smith FedML 109 4,496 0 21 Aug 2019
Neural News Recommendation with Attentive Multi-View Learning Chuhan Wu Fangzhao Wu Mingxiao An Jianqiang Huang Yongfeng Huang Xing Xie 45 277 0 12 Jul 2019
NPA: Neural News Recommendation with Personalized Attention Chuhan Wu Fangzhao Wu Mingxiao An Jianqiang Huang Yongfeng Huang Xing Xie AI4TS 53 323 0 12 Jul 2019
Deep Leakage from Gradients Ligeng Zhu Zhijian Liu Song Han FedML 90 2,199 0 21 Jun 2019
On Learning Over-parameterized Neural Networks: A Functional Approximation Perspective Lili Su Pengkun Yang MLT 51 53 0 26 May 2019
Neural Graph Collaborative Filtering Xiang Wang Xiangnan He Meng Wang Fuli Feng Tat-Seng Chua 154 2,965 0 20 May 2019
Distilling Task-Specific Knowledge from BERT into Simple Neural Networks Raphael Tang Yao Lu Linqing Liu Lili Mou Olga Vechtomova Jimmy J. Lin 61 419 0 28 Mar 2019
Expanding the Reach of Federated Learning by Reducing Client Resource Requirements S. Caldas Jakub Konecný H. B. McMahan Ameet Talwalkar 59 449 0 18 Dec 2018
Federated Learning for Mobile Keyboard Prediction Andrew Straiton Hard Kanishka Rao Zhifeng Lin Swaroop Indra Ramaswamy Youjie Li S. Augenstein Alex Schwing M. Annavaram A. Avestimehr FedML 129 1,531 0 08 Nov 2018
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.4K 94,511 0 11 Oct 2018
Large scale distributed neural network training through online distillation Rohan Anil Gabriel Pereyra Alexandre Passos Róbert Ormándi George E. Dahl Geoffrey E. Hinton FedML 317 407 0 09 Apr 2018
DKN: Deep Knowledge-Aware Network for News Recommendation Hongwei Wang Fuzheng Zhang Xing Xie Minyi Guo 65 1,043 0 25 Jan 2018
Scalable and accurate deep learning for electronic health records A. Rajkomar Eyal Oren Kai Chen Andrew M. Dai Nissan Hajaj ... A. Butte M. Howell Claire Cui Greg S. Corrado Jeffrey Dean OOD BDL 141 2,139 0 24 Jan 2018
Deep Mutual Learning Ying Zhang Tao Xiang Timothy M. Hospedales Huchuan Lu FedML 143 1,650 0 01 Jun 2017
Federated Learning: Strategies for Improving Communication Efficiency Jakub Konecný H. B. McMahan Felix X. Yu Peter Richtárik A. Suresh Dave Bacon FedML 286 4,636 0 18 Oct 2016
Communication-Efficient Learning of Deep Networks from Decentralized Data H. B. McMahan Eider Moore Daniel Ramage S. Hampson Blaise Agüera y Arcas FedML 375 17,399 0 17 Feb 2016
Distilling the Knowledge in a Neural Network Geoffrey E. Hinton Oriol Vinyals J. Dean FedML 304 19,580 0 09 Mar 2015
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 1.4K 149,842 0 22 Dec 2014
Convolutional Neural Networks for Sentence Classification Yoon Kim AILaw VLM 591 13,416 0 25 Aug 2014