Auto-Precision Scaling for Distributed Deep Learning

v1v2v3 (latest)

Auto-Precision Scaling for Distributed Deep Learning

20 November 2019

Yang You

ArXiv (abs)PDF HTML

Papers citing "Auto-Precision Scaling for Distributed Deep Learning"

13 / 13 papers shown

Title
QPyTorch: A Low-Precision Arithmetic Simulation Framework Tianyi Zhang Zhiqiu Lin Guandao Yang Christopher De Sa MQ 53 66 0 09 Oct 2019
A Study of BFLOAT16 for Deep Learning Training Dhiraj D. Kalamkar Dheevatsa Mudigere Naveen Mellempudi Dipankar Das K. Banerjee ... Sudarshan Srinivasan Abhisek Kundu M. Smelyanskiy Bharat Kaul Pradeep Dubey MQ 86 347 0 29 May 2019
Image Classification at Supercomputer Scale Chris Ying Sameer Kumar Dehao Chen Tao Wang Youlong Cheng VLM 51 123 0 16 Nov 2018
Rethinking floating point for deep learning Jeff Johnson MQ 109 139 0 01 Nov 2018
Horovod: fast and easy distributed deep learning in TensorFlow Alexander Sergeev Mike Del Balso 100 1,221 0 15 Feb 2018
Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training Chengyue Wu Song Han Huizi Mao Yu Wang W. Dally 148 1,409 0 05 Dec 2017
Extremely Large Minibatch SGD: Training ResNet-50 on ImageNet in 15 Minutes Takuya Akiba Shuji Suzuki Keisuke Fukuda VLM 52 314 0 12 Nov 2017
Flexpoint: An Adaptive Numerical Format for Efficient Training of Deep Neural Networks Urs Koster T. Webb Xin Eric Wang Marcel Nassar Arjun K. Bansal ... Luke Hornof A. Khosrowshahi Carey Kloss Ruby J. Pai N. Rao MQ 47 262 0 06 Nov 2017
Poseidon: An Efficient Communication Architecture for Distributed Deep Learning on GPU Clusters Huatian Zhang Zeyu Zheng Shizhen Xu Wei-Ming Dai Qirong Ho Xiaodan Liang Zhiting Hu Jinliang Wei P. Xie Eric Xing GNN 67 347 0 11 Jun 2017
Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour Priya Goyal Piotr Dollár Ross B. Girshick P. Noordhuis Lukasz Wesolowski Aapo Kyrola Andrew Tulloch Yangqing Jia Kaiming He 3DH 128 3,685 0 08 Jun 2017
TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning W. Wen Cong Xu Feng Yan Chunpeng Wu Yandan Wang Yiran Chen Hai Helen Li 163 990 0 22 May 2017
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 429 2,945 0 15 Sep 2016
The Cityscapes Dataset for Semantic Urban Scene Understanding Marius Cordts Mohamed Omran Sebastian Ramos Timo Rehfeld Markus Enzweiler Rodrigo Benenson Uwe Franke Stefan Roth Bernt Schiele 1.1K 11,644 0 06 Apr 2016

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.