Towards a Better Theoretical Understanding of Independent Subnetwork Training

28 June 2023

Papers citing "Towards a Better Theoretical Understanding of Independent Subnetwork Training"

18 / 18 papers shown

Title
FIARSE: Model-Heterogeneous Federated Learning via Importance-Aware Submodel Extraction Feijie Wu Xingchen Wang Yaqing Wang Tianci Liu Lu Su Jing Gao FedML 51 3 0 28 Jul 2024
MSfusion: A Dynamic Model Splitting Approach for Resource-Constrained Machines to Collaboratively Train Larger Models Jin Xie Songze Li FedML 44 0 0 04 Jul 2024
Sparser, Better, Deeper, Stronger: Improving Sparse Training with Exact Orthogonal Initialization A. Nowak Lukasz Gniecki Filip Szatkowski Jacek Tabor 37 0 0 03 Jun 2024
FedP3: Federated Personalized and Privacy-friendly Network Pruning under Model Heterogeneity Kai Yi Nidham Gazagnadou Peter Richtárik Lingjuan Lyu 79 11 0 15 Apr 2024
Efficient and Light-Weight Federated Learning via Asynchronous Distributed Dropout Chen Dun Mirian Hipolito Garcia C. Jermaine Dimitrios Dimitriadis Anastasios Kyrillidis 66 20 0 28 Oct 2022
EF21-P and Friends: Improved Theoretical Communication Complexity for Distributed Optimization with Bidirectional Compression Kaja Gruntkowska A. Tyurin Peter Richtárik 38 22 0 30 Sep 2022
Minibatch Stochastic Three Points Method for Unconstrained Smooth Minimization Soumia Boucherouite Grigory Malinovsky Peter Richtárik El Houcine Bergou 21 3 0 16 Sep 2022
Federated Pruning: Improving Neural Network Efficiency with Federated Learning Rongmei Lin Yonghui Xiao Tien-Ju Yang Ding Zhao Li Xiong Giovanni Motta Franccoise Beaufays FedML 39 12 0 14 Sep 2022
FL_PyTorch: optimization research simulator for federated learning Konstantin Burlachenko Samuel Horváth Peter Richtárik FedML 48 18 0 07 Feb 2022
Optimal Algorithms for Decentralized Stochastic Variational Inequalities D. Kovalev Aleksandr Beznosikov Abdurakhmon Sadiev Michael Persiianov Peter Richtárik Alexander Gasnikov 37 35 0 06 Feb 2022
Permutation Compressors for Provably Faster Distributed Nonconvex Optimization Rafal Szlendak A. Tyurin Peter Richtárik 133 35 0 07 Oct 2021
EF21 with Bells & Whistles: Practical Algorithmic Extensions of Modern Error Feedback Ilyas Fatkhullin Igor Sokolov Eduard A. Gorbunov Zhize Li Peter Richtárik 46 46 0 07 Oct 2021
Federated Dropout -- A Simple Approach for Enabling Federated Learning on Resource Constrained Devices Dingzhu Wen Ki-Jun Jeon Kaibin Huang FedML 70 90 0 30 Sep 2021
A Field Guide to Federated Optimization Jianyu Wang Zachary B. Charles Zheng Xu Gauri Joshi H. B. McMahan ... Mi Zhang Tong Zhang Chunxiang Zheng Chen Zhu Wennan Zhu FedML 187 412 0 14 Jul 2021
FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropout Samuel Horváth Stefanos Laskaridis Mario Almeida Ilias Leondiadis Stylianos I. Venieris Nicholas D. Lane 189 268 0 26 Feb 2021
Adaptivity of Stochastic Gradient Methods for Nonconvex Optimization Samuel Horváth Lihua Lei Peter Richtárik Michael I. Jordan 57 30 0 13 Feb 2020
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 245 1,833 0 17 Sep 2019
New Convergence Aspects of Stochastic Gradient Algorithms Lam M. Nguyen Phuong Ha Nguyen Peter Richtárik K. Scheinberg Martin Takáč Marten van Dijk 23 66 0 10 Nov 2018