PipeDream: Fast and Efficient Pipeline Parallel DNN Training

8 June 2018

Deepak Narayanan

Papers citing "PipeDream: Fast and Efficient Pipeline Parallel DNN Training"

23 / 23 papers shown

Title
A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics Kai He Rui Mao Qika Lin Yucheng Ruan Xiang Lan Mengling Feng Min Zhang LM&MA AILaw 148 166 0 28 Jan 2025
Hardware Scaling Trends and Diminishing Returns in Large-Scale Distributed Training Jared Fernandez Luca Wehrstedt Leonid Shamis Mostafa Elhoushi Kalyan Saladi Yonatan Bisk Emma Strubell Jacob Kahn 410 3 0 20 Nov 2024
DropBP: Accelerating Fine-Tuning of Large Language Models by Dropping Backward Propagation Sunghyeon Woo Baeseong Park Byeongwook Kim Minjung Jo S. Kwon Dongsuk Jeon Dongsoo Lee 84 2 0 27 Feb 2024
Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuning Jiasheng Ye Zaixiang Zheng Yu Bao Lihua Qian Quanquan Gu DiffM 94 16 0 23 Aug 2023
Device Placement Optimization with Reinforcement Learning Azalia Mirhoseini Hieu H. Pham Quoc V. Le Benoit Steiner Rasmus Larsen Yuefeng Zhou Naveen Kumar Mohammad Norouzi Samy Bengio J. Dean 48 438 0 13 Jun 2017
Poseidon: An Efficient Communication Architecture for Distributed Deep Learning on GPU Clusters Huatian Zhang Zeyu Zheng Shizhen Xu Wei-Ming Dai Qirong Ho Xiaodan Liang Zhiting Hu Jinliang Wei P. Xie Eric Xing GNN 50 343 0 11 Jun 2017
Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour Priya Goyal Piotr Dollár Ross B. Girshick P. Noordhuis Lukasz Wesolowski Aapo Kyrola Andrew Tulloch Yangqing Jia Kaiming He 3DH 89 3,666 0 08 Jun 2017
Omnivore: An Optimizer for Multi-device Deep Learning on CPUs and GPUs Stefan Hadjis Ce Zhang Ioannis Mitliagkas Dan Iter Christopher Ré 39 65 0 14 Jun 2016
TensorFlow: A system for large-scale machine learning Martín Abadi P. Barham Jianmin Chen Zhiwen Chen Andy Davis ... Vijay Vasudevan Pete Warden Martin Wicke Yuan Yu Xiaoqiang Zhang GNN AI4CE 317 18,300 0 27 May 2016
Revisiting Distributed Synchronous SGD Jianmin Chen Xinghao Pan R. Monga Samy Bengio Rafal Jozefowicz 59 799 0 04 Apr 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 1.3K 192,638 0 10 Dec 2015
MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems Tianqi Chen Mu Li Yutian Li Min Lin Naiyan Wang Minjie Wang Tianjun Xiao Bing Xu Chiyuan Zhang Zheng Zhang 100 2,243 0 03 Dec 2015
Sequence to Sequence -- Video to Text Subhashini Venugopalan Marcus Rohrbach Jeff Donahue Raymond J. Mooney Trevor Darrell Kate Saenko 91 1,417 0 03 May 2015
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift Sergey Ioffe Christian Szegedy OOD 279 43,154 0 11 Feb 2015
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 678 149,474 0 22 Dec 2014
LightLDA: Big Topic Models on Modest Compute Clusters Jinhui Yuan Fei Gao Qirong Ho Wei-Ming Dai Jinliang Wei Xun Zheng Eric Xing Tie-Yan Liu Wei-Ying Ma MoE AI4CE 107 177 0 04 Dec 2014
Show and Tell: A Neural Image Caption Generator Oriol Vinyals Alexander Toshev Samy Bengio D. Erhan 3DV 186 6,009 0 17 Nov 2014
Very Deep Convolutional Networks for Large-Scale Image Recognition Karen Simonyan Andrew Zisserman FAtt MDE 822 99,991 0 04 Sep 2014
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 986 39,383 0 01 Sep 2014
Caffe: Convolutional Architecture for Fast Feature Embedding Yangqing Jia Evan Shelhamer Jeff Donahue Sergey Karayev Jonathan Long Ross B. Girshick S. Guadarrama Trevor Darrell VLM BDL 3DV 179 14,703 0 20 Jun 2014
One weird trick for parallelizing convolutional neural networks A. Krizhevsky GNN 67 1,297 0 23 Apr 2014
A Convolutional Neural Network for Modelling Sentences Nal Kalchbrenner Edward Grefenstette Phil Blunsom 66 3,556 0 08 Apr 2014
Practical Bayesian Optimization of Machine Learning Algorithms Jasper Snoek Hugo Larochelle Ryan P. Adams 268 7,883 0 13 Jun 2012