Title
Hardware Scaling Trends and Diminishing Returns in Large-Scale Distributed Training Jared Fernandez Luca Wehrstedt Leonid Shamis Mostafa Elhoushi Kalyan Saladi Yonatan Bisk Emma Strubell Jacob Kahn 221 3 0 20 Nov 2024
AdaShadow: Responsive Test-time Model Adaptation in Non-stationary Mobile Environments Cheng Fang Sicong Liu Zimu Zhou Bin Guo Jiaqi Tang Ke Ma Zhiwen Yu TTA 36 1 0 10 Oct 2024
Partitioned Neural Network Training via Synthetic Intermediate Labels C. V. Karadag Nezih Topaloglu 34 1 0 17 Mar 2024
GraVAC: Adaptive Compression for Communication-Efficient Distributed DL Training S. Tyagi Martin Swany 25 4 0 20 May 2023
Optimus-CC: Efficient Large NLP Model Training with 3D Parallelism Aware Communication Compression Jaeyong Song Jinkyu Yim Jaewon Jung Hongsun Jang H. Kim Youngsok Kim Jinho Lee GNN 24 25 0 24 Jan 2023
Galvatron: Efficient Transformer Training over Multiple GPUs Using Automatic Parallelism Xupeng Miao Yujie Wang Youhe Jiang Chunan Shi Xiaonan Nie Hailin Zhang Bin Cui GNN MoE 42 60 0 25 Nov 2022
Distributed Graph Neural Network Training: A Survey Yingxia Shao Hongzheng Li Xizhi Gu Hongbo Yin Yawen Li Xupeng Miao Wentao Zhang Bin Cui Lei Chen GNN AI4CE 11 56 0 01 Nov 2022
PARTIME: Scalable and Parallel Processing Over Time with Deep Neural Networks Enrico Meloni Lapo Faggi Simone Marullo Alessandro Betti Matteo Tiezzi Marco Gori S. Melacci GNN AI4TS 19 1 0 17 Oct 2022
Communication-Efficient Adam-Type Algorithms for Distributed Data Mining Wenhan Xian Feihu Huang Heng-Chiao Huang FedML 35 0 0 14 Oct 2022
Optimizing DNN Compilation for Distributed Training with Joint OP and Tensor Fusion Xiaodong Yi Shiwei Zhang Lansong Diao Chuan Wu Zhen Zheng Shiqing Fan Siyu Wang Jun Yang W. Lin 39 4 0 26 Sep 2022
MLLess: Achieving Cost Efficiency in Serverless Machine Learning Training Pablo Gimeno Sarroca Marc Sánchez Artigas 24 14 0 12 Jun 2022
FuncPipe: A Pipelined Serverless Framework for Fast and Cost-efficient Training of Deep Learning Models Yunzhuo Liu Bo Jiang Tian Guo Zimeng Huang Wen-ping Ma Xinbing Wang Chenghu Zhou 24 9 0 28 Apr 2022
PICASSO: Unleashing the Potential of GPU-centric Training for Wide-and-deep Recommender Systems Yuanxing Zhang Langshi Chen Siran Yang Man Yuan Hui-juan Yi ... Yong Li Dingyang Zhang Wei Lin Lin Qu Bo Zheng 35 32 0 11 Apr 2022
HeterPS: Distributed Deep Learning With Reinforcement Learning Based Scheduling in Heterogeneous Environments Ji Liu Zhihua Wu Dianhai Yu Yanjun Ma Danlei Feng Minxu Zhang Xinxuan Wu Xuefeng Yao Dejing Dou 18 44 0 20 Nov 2021
Graph Neural Network Training with Data Tiering S. Min Kun Wu Mert Hidayetoğlu Jinjun Xiong Xiang Song Wen-mei W. Hwu GNN 25 15 0 10 Nov 2021
Colossal-AI: A Unified Deep Learning System For Large-Scale Parallel Training Yongbin Li Hongxin Liu Zhengda Bian Boxiang Wang Haichen Huang Fan Cui Chuan-Qing Wang Yang You GNN 30 143 0 28 Oct 2021
Taming Momentum in a Distributed Asynchronous Environment Ido Hakimi Saar Barkai Moshe Gabel Assaf Schuster 11 23 0 26 Jul 2019