How Can We Train Deep Learning Models Across Clouds and Continents? An Experimental Study

5 June 2023

Papers citing "How Can We Train Deep Learning Models Across Clouds and Continents? An Experimental Study"

4 / 4 papers shown

Title
Malleus: Straggler-Resilient Hybrid Parallel Training of Large-scale Models via Malleable Data and Model Parallelization Haoyang Li Fangcheng Fu Hao Ge Sheng Lin Xuanyu Wang Jiawen Niu Yufei Wang Hailin Zhang Xiaonan Nie Bin Cui MoMe 43 2 0 17 Oct 2024
Towards providing reliable job completion time predictions using PCS Abdullah Bin Faisal Noah Martin Hafiz Mohsin Bashir Swaminathan Lamelas Fahad R. Dogar 22 0 0 18 Jan 2024
A Survey on Efficient Federated Learning Methods for Foundation Model Training Herbert Woisetschläger Alexander Isenko Shiqiang Wang R. Mayer Hans-Arno Jacobsen FedML 65 24 0 09 Jan 2024
ZeRO-Offload: Democratizing Billion-Scale Model Training Jie Ren Samyam Rajbhandari Reza Yazdani Aminabadi Olatunji Ruwase Shuangyang Yang Minjia Zhang Dong Li Yuxiong He MoE 177 417 0 18 Jan 2021