Online Evolutionary Batch Size Orchestration for Scheduling Deep Learning Workloads in GPU Clusters

8 August 2021

Shenggui Li

Papers citing "Online Evolutionary Batch Size Orchestration for Scheduling Deep Learning Workloads in GPU Clusters"

27 / 27 papers shown

Title
Modeling Motion with Multi-Modal Features for Text-Based Video Segmentation Wangbo Zhao Kai Wang Xiangxiang Chu Fuzhao Xue Xinchao Wang Yang You 41 21 0 06 Apr 2022
Crafting Better Contrastive Views for Siamese Representation Learning Xiang Peng Kai Wang Zheng Hua Zhu Mang Wang Yang You SSL 58 104 0 07 Feb 2022
Go Wider Instead of Deeper Fuzhao Xue Ziji Shi Futao Wei Yuxuan Lou Yong Liu Yang You ViT MoE 35 81 0 25 Jul 2021
Concurrent Adversarial Learning for Large-Batch Training Yong Liu Xiangning Chen Minhao Cheng Cho-Jui Hsieh Yang You ODL 51 13 0 01 Jun 2021
Training EfficientNets at Supercomputer Scale: 83% ImageNet Top-1 Accuracy in One Hour Arissa Wongpanich Hieu H. Pham J. Demmel Mingxing Tan Quoc V. Le Yang You Sameer Kumar 34 8 0 30 Oct 2020
Auto-Precision Scaling for Distributed Deep Learning Ruobing Han J. Demmel Yang You 33 5 0 20 Nov 2019
DL2: A Deep Learning-driven Scheduler for Deep Learning Clusters Size Zheng Yixin Bao Yangrui Chen Chuan Wu Chen Meng Wei Lin 33 80 0 13 Sep 2019
Dynamic Mini-batch SGD for Elastic Distributed Training: Learning in the Limbo of Resources Yanghua Peng Hang Zhang Yifei Ma Tong He Zhi-Li Zhang Sheng Zha Mu Li 40 23 0 26 Apr 2019
Large Batch Optimization for Deep Learning: Training BERT in 76 minutes Yang You Jing Li Sashank J. Reddi Jonathan Hseu Sanjiv Kumar Srinadh Bhojanapalli Xiaodan Song J. Demmel Kurt Keutzer Cho-Jui Hsieh ODL 186 991 0 01 Apr 2019
Large-Batch Training for LSTM and Beyond Yang You Jonathan Hseu Chris Ying J. Demmel Kurt Keutzer Cho-Jui Hsieh 52 89 0 24 Jan 2019
Analysis of Large-Scale Multi-Tenant GPU Clusters for DNN Training Workloads Myeongjae Jeon Shivaram Venkataraman Amar Phanishayee Junjie Qian Wencong Xiao Fan Yang GNN 58 353 0 17 Jan 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.2K 93,936 0 11 Oct 2018
Learning Scheduling Algorithms for Data Processing Clusters Hongzi Mao Malte Schwarzkopf S. Venkatakrishnan Zili Meng Mohammad Alizadeh OffRL 76 642 0 03 Oct 2018
SLAQ: Quality-Driven Scheduling for Distributed Machine Learning Haoyu Zhang Logan Stafman Andrew Or M. Freedman 52 140 0 13 Feb 2018
Online Job Scheduling in Distributed Machine Learning Clusters Yixin Bao Size Zheng Chuan Wu Zongpeng Li 49 109 0 03 Jan 2018
Don't Decay the Learning Rate, Increase the Batch Size Samuel L. Smith Pieter-Jan Kindermans Chris Ying Quoc V. Le ODL 95 990 0 01 Nov 2017
Large Batch Training of Convolutional Networks Yang You Igor Gitman Boris Ginsburg ODL 116 844 0 13 Aug 2017
Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour Priya Goyal Piotr Dollár Ross B. Girshick P. Noordhuis Lukasz Wesolowski Aapo Kyrola Andrew Tulloch Yangqing Jia Kaiming He 3DH 107 3,666 0 08 Jun 2017
Train longer, generalize better: closing the generalization gap in large batch training of neural networks Elad Hoffer Itay Hubara Daniel Soudry ODL 150 799 0 24 May 2017
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 382 2,922 0 15 Sep 2016
TensorFlow: A system for large-scale machine learning Martín Abadi P. Barham Jianmin Chen Zhiwen Chen Andy Davis ... Vijay Vasudevan Pete Warden Martin Wicke Yuan Yu Xiaoqiang Zhang GNN AI4CE 368 18,300 0 27 May 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 1.6K 192,638 0 10 Dec 2015
Deep Speech 2: End-to-End Speech Recognition in English and Mandarin Dario Amodei Rishita Anubhai Eric Battenberg Carl Case Jared Casper ... Chong-Jun Wang Bo Xiao Dani Yogatama J. Zhan Zhenyao Zhu 116 2,965 0 08 Dec 2015
Rethinking the Inception Architecture for Computer Vision Christian Szegedy Vincent Vanhoucke Sergey Ioffe Jonathon Shlens Z. Wojna 3DV BDL 594 27,231 0 02 Dec 2015
Cyclical Learning Rates for Training Neural Networks L. Smith ODL 150 2,515 0 03 Jun 2015
Going Deeper with Convolutions Christian Szegedy Wei Liu Yangqing Jia P. Sermanet Scott E. Reed Dragomir Anguelov D. Erhan Vincent Vanhoucke Andrew Rabinovich 366 43,511 0 17 Sep 2014
Very Deep Convolutional Networks for Large-Scale Image Recognition Karen Simonyan Andrew Zisserman FAtt MDE 1.1K 99,991 0 04 Sep 2014