v1v2 (latest)

FedKD: Communication Efficient Federated Learning via Knowledge Distillation

30 August 2021

Xing Xie

Papers citing "FedKD: Communication Efficient Federated Learning via Knowledge Distillation"

42 / 42 papers shown

Title
LLM-QFL: Distilling Large Language Model for Quantum Federated Learning Dev Gurung Shiva Raj Pokhrel FedML 199 0 0 24 May 2025
Soft-Label Caching and Sharpening for Communication-Efficient Federated Distillation Kitsuya Azuma Takayuki Nishio Yuichi Kitagawa Wakako Nakano Takahito Tanimura FedML 163 0 0 28 Apr 2025
FedSA: A Unified Representation Learning via Semantic Anchors for Prototype-based Federated Learning Yanbing Zhou Xiangmou Qu Chenlong You Jiyang Zhou Jingyue Tang Xin Zheng Chunmao Cai Yingbo Wu FedML 121 3 0 09 Jan 2025
Explainable Semantic Federated Learning Enabled Industrial Edge Network for Fire Surveillance Li Dong Yubo Peng Feibo Jiang Kezhi Wang Kun Yang 85 1 0 31 Dec 2024
Overcoming Data and Model Heterogeneities in Decentralized Federated Learning via Synthetic Anchors Chun-Yin Huang Kartik Srinivas Xin Zhang Xiaoxiao Li DD 112 6 0 19 May 2024
Stochastic Controlled Averaging for Federated Learning with Communication Compression Xinmeng Huang Ping Li Xiaoyun Li 93 207 0 16 Aug 2023
When Foundation Model Meets Federated Learning: Motivations, Challenges, and Future Directions Weiming Zhuang Chen Chen Lingjuan Lyu Chong Chen Yaochu Jin Lingjuan Lyu AIFin AI4CE 175 96 0 27 Jun 2023
FedICT: Federated Multi-task Distillation for Multi-access Edge Computing Zhiyuan Wu Sheng Sun Yuwei Wang Min Liu Quyang Pan Xue Jiang Bo Gao 99 33 0 01 Jan 2023
Fast Federated Learning by Balancing Communication Trade-Offs Milad Khademi Nori Sangseok Yun Il-Min Kim FedML 59 55 0 23 May 2021
Pufferfish: Communication-efficient Models At No Extra Cost Hongyi Wang Saurabh Agarwal Dimitris Papailiopoulos 63 59 0 05 Mar 2021
Federated Knowledge Distillation Hyowoon Seo Jihong Park Seungeun Oh M. Bennis Seong-Lyun Kim FedML 77 92 0 04 Nov 2020
Federated Model Distillation with Noise-Free Differential Privacy Lichao Sun Lingjuan Lyu FedML 87 107 0 11 Sep 2020
FetchSGD: Communication-Efficient Federated Learning with Sketching D. Rothchild Ashwinee Panda Enayat Ullah Nikita Ivkin Ion Stoica Vladimir Braverman Joseph E. Gonzalez Raman Arora FedML 73 367 0 15 Jul 2020
Ensemble Distillation for Robust Model Fusion in Federated Learning Tao R. Lin Lingjing Kong Sebastian U. Stich Martin Jaggi FedML 102 1,041 0 12 Jun 2020
Pre-trained Models for Natural Language Processing: A Survey Xipeng Qiu Tianxiang Sun Yige Xu Yunfan Shao Ning Dai Xuanjing Huang LM&MA VLM 359 1,487 0 18 Mar 2020
UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training Hangbo Bao Li Dong Furu Wei Wenhui Wang Nan Yang ... Yu Wang Songhao Piao Jianfeng Gao Ming Zhou H. Hon AI4CE 86 393 0 28 Feb 2020
MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers Wenhui Wang Furu Wei Li Dong Hangbo Bao Nan Yang Ming Zhou VLM 156 1,267 0 25 Feb 2020
Advances and Open Problems in Federated Learning Peter Kairouz H. B. McMahan Brendan Avent A. Bellet M. Bennis ... Zheng Xu Qiang Yang Felix X. Yu Han Yu Sen Zhao FedML AI4CE 259 6,261 0 10 Dec 2019
FedMD: Heterogenous Federated Learning via Model Distillation Daliang Li Junpu Wang FedML 98 854 0 08 Oct 2019
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter Victor Sanh Lysandre Debut Julien Chaumond Thomas Wolf 232 7,520 0 02 Oct 2019
FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization Amirhossein Reisizadeh Aryan Mokhtari Hamed Hassani Ali Jadbabaie Ramtin Pedarsani FedML 254 774 0 28 Sep 2019
TinyBERT: Distilling BERT for Natural Language Understanding Xiaoqi Jiao Yichun Yin Lifeng Shang Xin Jiang Xiao Chen Linlin Li F. Wang Qun Liu VLM 107 1,860 0 23 Sep 2019
Patient Knowledge Distillation for BERT Model Compression S. Sun Yu Cheng Zhe Gan Jingjing Liu 134 837 0 25 Aug 2019
Federated Learning: Challenges, Methods, and Future Directions Tian Li Anit Kumar Sahu Ameet Talwalkar Virginia Smith FedML 123 4,517 0 21 Aug 2019
Neural News Recommendation with Attentive Multi-View Learning Chuhan Wu Fangzhao Wu Mingxiao An Jianqiang Huang Yongfeng Huang Xing Xie 49 277 0 12 Jul 2019
NPA: Neural News Recommendation with Personalized Attention Chuhan Wu Fangzhao Wu Mingxiao An Jianqiang Huang Yongfeng Huang Xing Xie AI4TS 60 323 0 12 Jul 2019
Deep Leakage from Gradients Ligeng Zhu Zhijian Liu Song Han FedML 97 2,207 0 21 Jun 2019
On Learning Over-parameterized Neural Networks: A Functional Approximation Perspective Lili Su Pengkun Yang MLT 65 54 0 26 May 2019
Neural Graph Collaborative Filtering Xiang Wang Xiangnan He Meng Wang Fuli Feng Tat-Seng Chua 187 2,981 0 20 May 2019
Distilling Task-Specific Knowledge from BERT into Simple Neural Networks Raphael Tang Yao Lu Linqing Liu Lili Mou Olga Vechtomova Jimmy J. Lin 69 421 0 28 Mar 2019
Expanding the Reach of Federated Learning by Reducing Client Resource Requirements S. Caldas Jakub Konecný H. B. McMahan Ameet Talwalkar 69 450 0 18 Dec 2018
Federated Learning for Mobile Keyboard Prediction Andrew Straiton Hard Kanishka Rao Zhifeng Lin Swaroop Indra Ramaswamy Youjie Li S. Augenstein Alex Schwing M. Annavaram A. Avestimehr FedML 136 1,541 0 08 Nov 2018
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 94,891 0 11 Oct 2018
Large scale distributed neural network training through online distillation Rohan Anil Gabriel Pereyra Alexandre Passos Róbert Ormándi George E. Dahl Geoffrey E. Hinton FedML 320 408 0 09 Apr 2018
DKN: Deep Knowledge-Aware Network for News Recommendation Hongwei Wang Fuzheng Zhang Xing Xie Minyi Guo 75 1,045 0 25 Jan 2018
Scalable and accurate deep learning for electronic health records A. Rajkomar Eyal Oren Kai Chen Andrew M. Dai Nissan Hajaj ... A. Butte M. Howell Claire Cui Greg S. Corrado Jeffrey Dean OOD BDL 168 2,145 0 24 Jan 2018
Deep Mutual Learning Ying Zhang Tao Xiang Timothy M. Hospedales Huchuan Lu FedML 151 1,653 0 01 Jun 2017
Federated Learning: Strategies for Improving Communication Efficiency Jakub Konecný H. B. McMahan Felix X. Yu Peter Richtárik A. Suresh Dave Bacon FedML 306 4,646 0 18 Oct 2016
Communication-Efficient Learning of Deep Networks from Decentralized Data H. B. McMahan Eider Moore Daniel Ramage S. Hampson Blaise Agüera y Arcas FedML 406 17,486 0 17 Feb 2016
Distilling the Knowledge in a Neural Network Geoffrey E. Hinton Oriol Vinyals J. Dean FedML 362 19,660 0 09 Mar 2015
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 1.9K 150,115 0 22 Dec 2014
Convolutional Neural Networks for Sentence Classification Yoon Kim AILaw VLM 635 13,427 0 25 Aug 2014