Horovod: fast and easy distributed deep learning in TensorFlow

15 February 2018

Papers citing "Horovod: fast and easy distributed deep learning in TensorFlow"

50 / 174 papers shown

Title
Clairvoyant Prefetching for Distributed Machine Learning I/O Nikoli Dryden Roman Böhringer Tal Ben-Nun Torsten Hoefler 36 57 0 21 Jan 2021
SceneGen: Learning to Generate Realistic Traffic Scenes Shuhan Tan K. Wong Shenlong Wang S. Manivasagam Mengye Ren R. Urtasun 26 100 0 16 Jan 2021
NeurST: Neural Speech Translation Toolkit Chengqi Zhao Mingxuan Wang Qianqian Dong Rong Ye Lei Li 30 32 0 18 Dec 2020
Data optimization for large batch distributed training of deep neural networks Shubhankar Gahlot Junqi Yin Mallikarjun Shankar 21 1 0 16 Dec 2020
A Study of Checkpointing in Large Scale Training of Deep Neural Networks Elvis Rojas A. Kahira Esteban Meneses L. Bautista-Gomez Rosa M. Badia 29 22 0 01 Dec 2020
Protein model quality assessment using rotation-equivariant, hierarchical neural networks Stephan Eismann Patricia Suriana Bowen Jing Raphael J. L. Townshend R. Dror 9 13 0 27 Nov 2020
TLeague: A Framework for Competitive Self-Play based Distributed Multi-Agent Reinforcement Learning Peng Sun Jiechao Xiong Lei Han Xinghai Sun Shuxing Li Jiawei Xu Meng Fang Zhengyou Zhang OffRL LRM 33 19 0 25 Nov 2020
Integrating Deep Learning in Domain Sciences at Exascale Rick Archibald E. Chow E. DÁzevedo Jack J. Dongarra M. Eisenbach ... Florent Lopez Daniel Nichols S. Tomov Kwai Wong Junqi Yin PINN 23 5 0 23 Nov 2020
A Novel Memory-Efficient Deep Learning Training Framework via Error-Bounded Lossy Compression Sian Jin Guanpeng Li Shuaiwen Leon Song Dingwen Tao AI4CE 29 12 0 18 Nov 2020
Context-Aware Drive-thru Recommendation Service at Fast Food Restaurants Luyang Wang Kai-Qi Huang Jiao Wang Shengsheng Huang J. Dai Zhuang Yue 22 1 0 13 Oct 2020
DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphs Da Zheng Chao Ma Minjie Wang Jinjing Zhou Qidong Su Xiang Song Quan Gan Zheng-Wei Zhang George Karypis FedML GNN 27 243 0 11 Oct 2020
Accelerating Finite-temperature Kohn-Sham Density Functional Theory with Deep Neural Networks J. Ellis Lenz Fiedler G. Popoola N. Modine J. A. Stephens A. Thompson A. Cangi S. Rajamanickam AI4CE 32 40 0 10 Oct 2020
A Tensor Compiler for Unified Machine Learning Prediction Serving Supun Nakandala Karla Saur Karla Saur Gyeong-In Yu Konstantinos Karanasos Carlo Curino Markus Weimer Matteo Interlandi 37 53 0 09 Oct 2020
Towards a Scalable and Distributed Infrastructure for Deep Learning Applications Bita Hasheminezhad S. Shirzad Nanmiao Wu Patrick Diehl Hannes Schulz Hartmut Kaiser GNN AI4CE 32 4 0 06 Oct 2020
VirtualFlow: Decoupling Deep Learning Models from the Underlying Hardware Andrew Or Haoyu Zhang M. Freedman 19 9 0 20 Sep 2020
Looking for change? Roll the Dice and demand Attention F. Diakogiannis F. Waldner P. Caccetta 23 66 0 04 Sep 2020
Accuracy and Performance Comparison of Video Action Recognition Approaches Matthew Hutchinson S. Samsi William Arcand David Bestor Bill Bergeron ... Andrew Prout Antonio Rosa Albert Reuther Charles Yee V. Gadepally 16 5 0 20 Aug 2020
Synthesizing Optimal Collective Algorithms Zixian Cai Zhengyang Liu Saeed Maleki Madan Musuvathi Todd Mytkowicz Jacob Nelson Olli Saarikivi GNN 26 60 0 19 Aug 2020
A Survey on Large-scale Machine Learning Meng Wang Weijie Fu Xiangnan He Shijie Hao Xindong Wu 25 110 0 10 Aug 2020
Temporal Context Aggregation for Video Retrieval with Contrastive Learning Jie Shao Xin Wen Bingchen Zhao Xiangyang Xue AI4TS 38 4 0 04 Aug 2020
The Case for Strong Scaling in Deep Learning: Training Large 3D CNNs with Hybrid Parallelism Yosuke Oyama N. Maruyama Nikoli Dryden Erin McCarthy P. Harrington J. Balewski Satoshi Matsuoka Peter Nugent B. Van Essen 3DV AI4CE 37 37 0 25 Jul 2020
HyperTune: Dynamic Hyperparameter Tuning For Efficient Distribution of DNN Training Over Heterogeneous Systems Ali Heydarigorji Siavash Rezaei Mahdi Torabzadehkashi Hossein Bobarshad V. Alves Pai H. Chou 24 14 0 16 Jul 2020
DAPPLE: A Pipelined Data Parallel Approach for Training Large Models Shiqing Fan Yi Rong Chen Meng Zongyan Cao Siyu Wang ... Jun Yang Lixue Xia Lansong Diao Xiaoyong Liu Wei Lin 21 233 0 02 Jul 2020
Is Network the Bottleneck of Distributed Training? Zhen Zhang Chaokun Chang Yanghua Peng Yida Wang R. Arora Xin Jin 25 70 0 17 Jun 2020
FinBERT: A Pretrained Language Model for Financial Communications Yi Yang Mark Christopher Siy Uy Allen H Huang AIFin AI4CE 28 231 0 15 Jun 2020
A Scalable and Cloud-Native Hyperparameter Tuning System Johnu George Ce Gao Richard Liu Hou Gang Liu Yuan Tang Ramdoot Pydipaty Amit Kumar Saha 13 13 0 03 Jun 2020
HetPipe: Enabling Large DNN Training on (Whimpy) Heterogeneous GPU Clusters through Integration of Pipelined Model Parallelism and Data Parallelism Jay H. Park Gyeongchan Yun Chang Yi N. T. Nguyen Seungmin Lee Jaesik Choi S. Noh Young-ri Choi MoE 25 129 0 28 May 2020
Bayesian Neural Networks at Scale: A Performance Analysis and Pruning Study Himanshu Sharma Elise Jennings BDL 27 3 0 23 May 2020
Deploying Scientific AI Networks at Petaflop Scale on Secure Large Scale HPC Production Systems with Containers D. Brayford S. Vallecorsa 14 8 0 20 May 2020
Tropical and Extratropical Cyclone Detection Using Deep Learning Christina Kumler-Bonfanti J. Stewart D. Hall M. Govett 24 37 0 18 May 2020
TensorOpt: Exploring the Tradeoffs in Distributed DNN Training with Auto-Parallelism Zhenkun Cai Kaihao Ma Xiao Yan Yidi Wu Yuzhen Huang James Cheng Teng Su F. Yu 24 42 0 16 Apr 2020
Characterizing and Modeling Distributed Training with Transient Cloud GPU Servers Shijian Li R. Walls Tian Guo 31 23 0 07 Apr 2020
Fiber: A Platform for Efficient Development and Distributed Training for Reinforcement Learning and Population-Based Methods Jiale Zhi Rui Wang Jeff Clune Kenneth O. Stanley OffRL 30 12 0 25 Mar 2020
Communication optimization strategies for distributed deep neural network training: A survey Shuo Ouyang Dezun Dong Yemao Xu Liquan Xiao 30 12 0 06 Mar 2020
Communication Contention Aware Scheduling of Multiple Deep Learning Training Jobs Qiang-qiang Wang Shaoshuai Shi Canhui Wang Xiaowen Chu 26 13 0 24 Feb 2020
Communication-Efficient Edge AI: Algorithms and Systems Yuanming Shi Kai Yang Tao Jiang Jun Zhang Khaled B. Letaief GNN 29 327 0 22 Feb 2020
Hoplite: Efficient and Fault-Tolerant Collective Communication for Task-Based Distributed Systems Siyuan Zhuang Zhuohan Li Danyang Zhuo Stephanie Wang Eric Liang Robert Nishihara Philipp Moritz Ion Stoica 27 23 0 13 Feb 2020
Stochastic Weight Averaging in Parallel: Large-Batch Training that Generalizes Well Vipul Gupta S. Serrano D. DeCoste MoMe 38 55 0 07 Jan 2020
A Survey on Distributed Machine Learning Joost Verbraeken Matthijs Wolting Jonathan Katzy Jeroen Kloppenburg Tim Verbelen Jan S. Rellermeyer OOD 42 692 0 20 Dec 2019
C2FNAS: Coarse-to-Fine Neural Architecture Search for 3D Medical Image Segmentation Qihang Yu Dong Yang H. Roth Yutong Bai Yixiao Zhang Alan Yuille Daguang Xu 37 108 0 20 Dec 2019
Understanding Top-k Sparsification in Distributed Deep Learning Shaoshuai Shi Xiaowen Chu Ka Chun Cheung Simon See 30 95 0 20 Nov 2019
Label-similarity Curriculum Learning Ürün Dogan A. Deshmukh Marcin Machura Christian Igel 23 21 0 15 Nov 2019
Highly-scalable, physics-informed GANs for learning solutions of stochastic PDEs Liu Yang Sean Treichler Thorsten Kurth Keno Fischer D. Barajas-Solano ... Valentin Churavy A. Tartakovsky Michael Houston P. Prabhat George Karniadakis AI4CE 47 38 0 29 Oct 2019
Parallelized Training of Restricted Boltzmann Machines using Markov-Chain Monte Carlo Methods Pei Yang S. Varadharajan Lucas A. Wilson Don D. Smith John A. Lockman Vineet Gundecha Quy Ta BDL 17 1 0 14 Oct 2019
Blink: Fast and Generic Collectives for Distributed ML Guanhua Wang Shivaram Venkataraman Amar Phanishayee J. Thelin Nikhil R. Devanur Ion Stoica VLM 17 136 0 11 Oct 2019
Distributed Learning of Deep Neural Networks using Independent Subnet Training John Shelton Hyatt Cameron R. Wolfe Michael Lee Yuxin Tang Anastasios Kyrillidis Christopher M. Jermaine OOD 29 35 0 04 Oct 2019
Deep learning at scale for subgrid modeling in turbulent flows Mathis Bode M. Gauding K. Kleinheinz H. Pitsch AI4CE 21 21 0 01 Oct 2019
Exascale Deep Learning for Scientific Inverse Problems N. Laanait Josh Romero Junqi Yin M. T. Young Sean Treichler V. Starchenko A. Borisevich Alexander Sergeev Michael A. Matheson FedML BDL 35 29 0 24 Sep 2019
Distributed Deep Learning for Precipitation Nowcasting S. Samsi Christopher J. Mattioli Mark S. Veillette 35 23 0 28 Aug 2019
Optimizing Multi-GPU Parallelization Strategies for Deep Learning Training Saptadeep Pal Eiman Ebrahimi A. Zulfiqar Yaosheng Fu Victor Zhang Szymon Migacz D. Nellans Puneet Gupta 36 55 0 30 Jul 2019