Unity is Power: Semi-Asynchronous Collaborative Training of Large-Scale
Models with Structured Pruning in Resource-Limited Clients

Unity is Power: Semi-Asynchronous Collaborative Training of Large-Scale Models with Structured Pruning in Resource-Limited Clients

11 October 2024

Xiao Zhang

ArXiv (abs)PDF HTML

Papers citing "Unity is Power: Semi-Asynchronous Collaborative Training of Large-Scale Models with Structured Pruning in Resource-Limited Clients"

19 / 19 papers shown

Title
TimelyFL: Heterogeneity-aware Asynchronous Federated Learning with Adaptive Partial Training Tuo Zhang Lei Gao Sunwoo Lee Mi Zhang Salman Avestimehr FedML 79 30 0 14 Apr 2023
Quantized Distributed Training of Large Models with Convergence Guarantees I. Markov Adrian Vladu Qi Guo Dan Alistarh MQ 77 11 0 05 Feb 2023
Adaptive Control of Client Selection and Gradient Compression for Efficient Federated Learning Zhida Jiang Yang Xu Hong-Ze Xu Zhiyuan Wang Chen Qian 46 9 0 19 Dec 2022
Can Foundation Models Perform Zero-Shot Task Specification For Robot Manipulation? Yuchen Cui S. Niekum Abhi Gupta Vikash Kumar Aravind Rajeswaran LM&Ro 86 79 0 23 Apr 2022
Q-ViT: Fully Differentiable Quantization for Vision Transformer Zhexin Li Tong Yang Peisong Wang Jian Cheng ViT MQ 84 42 0 19 Jan 2022
Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey Bonan Min Hayley L Ross Elior Sulem Amir Pouran Ben Veyseh Thien Huu Nguyen Oscar Sainz Eneko Agirre Ilana Heinz Dan Roth LM&MA VLM AI4CE 165 1,081 0 01 Nov 2021
Asynchronous Federated Learning on Heterogeneous Devices: A Survey Chenhao Xu Youyang Qu Yong Xiang Longxiang Gao FedML 160 258 0 09 Sep 2021
QuPeD: Quantized Personalization via Distillation with Applications to Federated Learning Kaan Ozkara Navjot Singh Deepesh Data Suhas Diggavi FedML MQ 82 58 0 29 Jul 2021
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 670 41,430 0 22 Oct 2020
FedAT: A High-Performance and Communication-Efficient Federated Learning System with Asynchronous Tiers Zheng Chai Yujing Chen Ali Anwar Liang Zhao Yue Cheng Huzefa Rangwala FedML 58 124 0 12 Oct 2020
Federated Learning with Compression: Unified Analysis and Sharp Guarantees Farzin Haddadpour Mohammad Mahdi Kamani Aryan Mokhtari M. Mahdavi FedML 84 279 0 02 Jul 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 877 42,379 0 28 May 2020
A Double Residual Compression Algorithm for Efficient Distributed Learning Xiaorui Liu Yao Li Jiliang Tang Ming Yan 54 50 0 16 Oct 2019
Model Pruning Enables Efficient Federated Learning on Edge Devices Yuang Jiang Shiqiang Wang Victor Valls Bongjun Ko Wei-Han Lee Kin K. Leung Leandros Tassiulas 92 463 0 26 Sep 2019
Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification T. Hsu Qi Matthew Brown FedML 143 1,163 0 13 Sep 2019
Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation Linfeng Zhang Jiebo Song Anni Gao Jingwei Chen Chenglong Bao Kaisheng Ma FedML 76 865 0 17 May 2019
Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask Hattie Zhou Janice Lan Rosanne Liu J. Yosinski UQCV 58 389 0 03 May 2019
Communication-Efficient Learning of Deep Networks from Decentralized Data H. B. McMahan Eider Moore Daniel Ramage S. Hampson Blaise Agüera y Arcas FedML 408 17,593 0 17 Feb 2016
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 1.7K 39,595 0 01 Sep 2014

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.