Dynamic Mini-batch SGD for Elastic Distributed Training: Learning in the
Limbo of Resources

Dynamic Mini-batch SGD for Elastic Distributed Training: Learning in the Limbo of Resources

26 April 2019

Sheng Zha

Papers citing "Dynamic Mini-batch SGD for Elastic Distributed Training: Learning in the Limbo of Resources"

6 / 6 papers shown

Title
GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection Jiawei Zhao Zhenyu (Allen) Zhang Beidi Chen Zhangyang Wang A. Anandkumar Yuandong Tian 43 175 0 06 Mar 2024
An Optimal Resource Allocator of Elastic Training for Deep Learning Jobs on Cloud Liang Hu Jiangcheng Zhu Zirui Zhou Ruiqing Cheng Xiaolong Bai Yong Zhang 16 3 0 08 Sep 2021
Online Evolutionary Batch Size Orchestration for Scheduling Deep Learning Workloads in GPU Clusters Chen Sun Shenggui Li Jinyue Wang Jun Yu 54 47 0 08 Aug 2021
AdaScale SGD: A User-Friendly Algorithm for Distributed Training Tyler B. Johnson Pulkit Agrawal Haijie Gu Carlos Guestrin ODL 24 37 0 09 Jul 2020
ResNeSt: Split-Attention Networks Hang Zhang Chongruo Wu Zhongyue Zhang Yi Zhu Yanghua Peng ... Tong He Jonas W. Mueller R. Manmatha Mu Li Alex Smola 51 1,462 0 19 Apr 2020
Bag of Tricks for Image Classification with Convolutional Neural Networks Tong He Zhi-Li Zhang Hang Zhang Zhongyue Zhang Junyuan Xie Mu Li 221 1,399 0 04 Dec 2018