v1v2v3 (latest)

Horovod: fast and easy distributed deep learning in TensorFlow

15 February 2018

Papers citing "Horovod: fast and easy distributed deep learning in TensorFlow"

50 / 454 papers shown

Title
Large-scale Neural Solvers for Partial Differential Equations Patrick Stiller Friedrich Bethke M. Böhme R. Pausch Sunna Torge A. Debus J. Vorberger Michael Bussmann Nico Hoffmann AI4CE 73 26 0 08 Sep 2020
S-SGD: Symmetrical Stochastic Gradient Descent with Weight Noise Injection for Reaching Flat Minima Wonyong Sung Iksoo Choi Jinhwan Park Seokhyun Choi Sungho Shin ODL 58 7 0 05 Sep 2020
Looking for change? Roll the Dice and demand Attention F. Diakogiannis F. Waldner P. Caccetta 121 68 0 04 Sep 2020
Pollux: Co-adaptive Cluster Scheduling for Goodput-Optimized Deep Learning Aurick Qiao Sang Keun Choe Suhas Jayaram Subramanya Willie Neiswanger Qirong Ho Hao Zhang G. Ganger Eric Xing VLM 77 183 0 27 Aug 2020
Accuracy and Performance Comparison of Video Action Recognition Approaches Matthew Hutchinson S. Samsi William Arcand David Bestor Bill Bergeron ... Andrew Prout Antonio Rosa Albert Reuther Charles Yee V. Gadepally 42 5 0 20 Aug 2020
Synthesizing Optimal Collective Algorithms Zixian Cai Zhengyang Liu Saeed Maleki Madan Musuvathi Todd Mytkowicz Jacob Nelson Olli Saarikivi GNN 84 61 0 19 Aug 2020
Domain-specific Communication Optimization for Distributed DNN Training Hao Wang Jingrong Chen Xinchen Wan Han Tian Jiacheng Xia Gaoxiong Zeng Weiyan Wang Kai Chen Wei Bai Junchen Jiang AI4CE 26 16 0 16 Aug 2020
Addressing Class Imbalance in Federated Learning Lixu Wang Shichao Xu Tianlin Li Qi Zhu FedML 67 18 0 14 Aug 2020
The Sockeye 2 Neural Machine Translation Toolkit at AMTA 2020 Tobias Domhan Michael J. Denkowski David Vilar Xing Niu Felix Hieber Kenneth Heafield 69 52 0 11 Aug 2020
A Survey on Large-scale Machine Learning Meng Wang Weijie Fu Xiangnan He Shijie Hao Xindong Wu 84 112 0 10 Aug 2020
Temporal Context Aggregation for Video Retrieval with Contrastive Learning Jie Shao Xin Wen Bingchen Zhao Xiangyang Xue AI4TS 93 4 0 04 Aug 2020
Flower: A Friendly Federated Learning Research Framework Daniel J. Beutel Taner Topal Akhil Mathur Xinchi Qiu Javier Fernandez-Marques ... Lorenzo Sani Kwing Hei Li Titouan Parcollet Pedro Porto Buarque de Gusmão Nicholas D. Lane FedML 158 825 0 28 Jul 2020
HeAT -- a Distributed and GPU-accelerated Tensor Framework for Data Analytics Markus Goetz D. Coquelin Charlotte Debus K. Krajsek C. Comito ... Michael Tarnawa Simon Hanselmann M. Siggel A. Basermann Achim Streit 34 9 0 27 Jul 2020
FedML: A Research Library and Benchmark for Federated Machine Learning Chaoyang He Songze Li Jinhyun So Xiao Zeng Mi Zhang ... Yang Liu Ramesh Raskar Qiang Yang M. Annavaram Salman Avestimehr FedML 269 578 0 27 Jul 2020
CSER: Communication-efficient SGD with Error Reset Cong Xie Shuai Zheng Oluwasanmi Koyejo Indranil Gupta Mu Li Yanghua Peng 108 40 0 26 Jul 2020
The Case for Strong Scaling in Deep Learning: Training Large 3D CNNs with Hybrid Parallelism Yosuke Oyama N. Maruyama Nikoli Dryden Erin McCarthy P. Harrington J. Balewski Satoshi Matsuoka Peter Nugent B. Van Essen 3DV AI4CE 71 37 0 25 Jul 2020
Deep Generative Models that Solve PDEs: Distributed Computing for Training Large Data-Free Models Sergio Botelho Ameya Joshi Biswajit Khara Soumik Sarkar Chinmay Hegde Santi S. Adavani Baskar Ganapathysubramanian AI4CE 52 6 0 24 Jul 2020
Computational Causal Inference Jeffrey Wong CML AI4CE 44 11 0 21 Jul 2020
High Performance Data Engineering Everywhere Chathura Widanage Niranda Perera V. Abeykoon Supun Kamburugamuve Thejaka Amila Kanewala Hasara Maithree P. Wickramasinghe A. Uyar Gurhan Gunduz Geoffrey C. Fox 38 18 0 19 Jul 2020
HyperTune: Dynamic Hyperparameter Tuning For Efficient Distribution of DNN Training Over Heterogeneous Systems Ali Heydarigorji Siavash Rezaei Mahdi Torabzadehkashi Hossein Bobarshad V. Alves Pai H. Chou 82 14 0 16 Jul 2020
ProtTrans: Towards Cracking the Language of Life's Code Through Self-Supervised Deep Learning and High Performance Computing Ahmed Elnaggar M. Heinzinger Christian Dallago Ghalia Rehawi Yu Wang ... Tamas B. Fehér Christoph Angerer Martin Steinegger D. Bhowmik B. Rost DRL 89 968 0 13 Jul 2020
DAPPLE: A Pipelined Data Parallel Approach for Training Large Models Shiqing Fan Yi Rong Chen Meng Zongyan Cao Siyu Wang ... Jun Yang Lixue Xia Lansong Diao Xiaoyong Liu Wei Lin 96 242 0 02 Jul 2020
Convolutional Neural Network Training with Distributed K-FAC J. G. Pauloski Zhao Zhang Lei Huang Weijia Xu Ian Foster 59 31 0 01 Jul 2020
Shuffle-Exchange Brings Faster: Reduce the Idle Time During Communication for Decentralized Neural Network Training Xiang Yang FedML 20 2 0 01 Jul 2020
PyTorch Distributed: Experiences on Accelerating Data Parallel Training Shen Li Yanli Zhao R. Varma Omkar Salpekar P. Noordhuis ... Adam Paszke Jeff Smith Brian Vaughan Pritam Damania Soumith Chintala OOD MoE 80 190 0 28 Jun 2020
Microstructure Generation via Generative Adversarial Network for Heterogeneous, Topologically Complex 3D Materials Tim Hsu W. Epting Hokon Kim H. Abernathy Gregory A. Hackett A. Rollett P. Salvador Elizabeth A. Holm 35 84 0 22 Jun 2020
Is Network the Bottleneck of Distributed Training? Zhen Zhang Chaokun Chang Yanghua Peng Yida Wang R. Arora Xin Jin 89 71 0 17 Jun 2020
Improving accuracy and speeding up Document Image Classification through parallel systems Javier Ferrando J. L. Domínguez Jordi Torres Raul Garcia David García Daniel Garrido J. Cortada M. Valero 119 26 0 16 Jun 2020
FinBERT: A Pretrained Language Model for Financial Communications Yi Yang Mark Christopher Siy Uy Allen H Huang AIFin AI4CE 85 238 0 15 Jun 2020
O(1) Communication for Distributed SGD through Two-Level Gradient Averaging Subhadeep Bhattacharya Weikuan Yu Fahim Chowdhury FedML 26 2 0 12 Jun 2020
Hierarchical, rotation-equivariant neural networks to select structural models of protein complexes Stephan Eismann Raphael J. L. Townshend Nathaniel Thomas Milind Jagota Bowen Jing R. Dror 50 8 0 05 Jun 2020
A Linear Algebraic Approach to Model Parallelism in Deep Learning Russell J. Hewett Thomas J. Grady FedML 47 16 0 04 Jun 2020
A Scalable and Cloud-Native Hyperparameter Tuning System Johnu George Ce Gao Richard Liu Hou Gang Liu Yuan Tang Ramdoot Pydipaty Amit Kumar Saha 17 13 0 03 Jun 2020
Parallelizing Machine Learning as a Service for the End-User Daniela Loreti Marco Lippi Paolo Torroni FedML 52 9 0 28 May 2020
HetPipe: Enabling Large DNN Training on (Whimpy) Heterogeneous GPU Clusters through Integration of Pipelined Model Parallelism and Data Parallelism Jay H. Park Gyeongchan Yun Chang Yi N. T. Nguyen Seungmin Lee Jaesik Choi S. Noh Young-ri Choi MoE 89 134 0 28 May 2020
A Comparative Study of Machine Learning Models for Tabular Data Through Challenge of Monitoring Parkinson's Disease Progression Using Voice Recordings Mohammadreza Iman Amy Giuntini H. Arabnia Khaled Rasheed 28 2 0 27 May 2020
Bayesian Neural Networks at Scale: A Performance Analysis and Pruning Study Himanshu Sharma Elise Jennings BDL 75 4 0 23 May 2020
Deploying Scientific AI Networks at Petaflop Scale on Secure Large Scale HPC Production Systems with Containers D. Brayford S. Vallecorsa 48 8 0 20 May 2020
Tropical and Extratropical Cyclone Detection Using Deep Learning Christina Kumler-Bonfanti J. Stewart D. Hall M. Govett 36 37 0 18 May 2020
Scaling-up Distributed Processing of Data Streams for Machine Learning M. Nokleby Haroon Raja W. Bajwa 69 15 0 18 May 2020
OD-SGD: One-step Delay Stochastic Gradient Descent for Distributed Training Yemao Xu Dezun Dong Weixia Xu Xiangke Liao 47 7 0 14 May 2020
Benchmark Tests of Convolutional Neural Network and Graph Convolutional Network on HorovodRunner Enabled Spark Clusters Jing Pan Wendao Liu Jing Zhou GNN BDL 25 2 0 12 May 2020
Breaking (Global) Barriers in Parallel Stochastic Optimization with Wait-Avoiding Group Averaging Shigang Li Tal Ben-Nun Giorgi Nadiradze Salvatore Di Girolamo Nikoli Dryden Dan Alistarh Torsten Hoefler 75 15 0 30 Apr 2020
Caramel: Accelerating Decentralized Distributed Deep Learning with Computation Scheduling Sayed Hadi Hashemi Sangeetha Abdu Jyothi Brighten Godfrey R. Campbell 34 2 0 29 Apr 2020
TensorOpt: Exploring the Tradeoffs in Distributed DNN Training with Auto-Parallelism Zhenkun Cai Kaihao Ma Xiao Yan Yidi Wu Yuzhen Huang James Cheng Teng Su F. Yu 66 45 0 16 Apr 2020
Characterizing and Modeling Distributed Training with Transient Cloud GPU Servers Shijian Li R. Walls Tian Guo 55 23 0 07 Apr 2020
Fiber: A Platform for Efficient Development and Distributed Training for Reinforcement Learning and Population-Based Methods Jiale Zhi Rui Wang Jeff Clune Kenneth O. Stanley OffRL 72 12 0 25 Mar 2020
Convergence of Artificial Intelligence and High Performance Computing on NSF-supported Cyberinfrastructure Eliu A. Huerta Asad Khan Edward Davis Colleen Bushell W. Gropp ... S. Koric William T. C. Kramer Brendan McGinty Kenton McHenry Aaron Saxton AI4CE 107 45 0 18 Mar 2020
Communication optimization strategies for distributed deep neural network training: A survey Shuo Ouyang Dezun Dong Yemao Xu Liquan Xiao 123 12 0 06 Mar 2020
Do optimization methods in deep learning applications matter? Buse Melis Özyildirim Mariam Kiran 33 11 0 28 Feb 2020