v1v2v3 (latest)

Horovod: fast and easy distributed deep learning in TensorFlow

15 February 2018

Papers citing "Horovod: fast and easy distributed deep learning in TensorFlow"

50 / 454 papers shown

Title
Learning Electron Bunch Distribution along a FEL Beamline by Normalising Flows Anna Willmann J. C. Cabadağ Yen-Yu Chang R. Pausch Amin Ghaith A. Debus A. Irman Michael Bussmann Steve Schmerler Nico Hoffmann 35 0 0 27 Feb 2023
FLINT: A Platform for Federated Learning Integration Ewen N. Wang Ajaykumar Kannan Yuefeng Liang Boyi Chen Mosharaf Chowdhury 102 24 0 24 Feb 2023
DeAR: Accelerating Distributed Deep Learning with Fine-Grained All-Reduce Pipelining Lin Zhang Shaoshuai Shi Xiaowen Chu Wei Wang Yue Liu Chengjian Liu 75 11 0 24 Feb 2023
h-analysis and data-parallel physics-informed neural networks Paul Escapil-Inchauspé G. A. Ruz PINN AI4CE 65 3 0 17 Feb 2023
CarbonScaler: Leveraging Cloud Workload Elasticity for Optimizing Carbon-Efficiency Walid A. Hanafy Qianlin Liang Noman Bashir David Irwin Prashant J. Shenoy 77 52 0 17 Feb 2023
THC: Accelerating Distributed Deep Learning Using Tensor Homomorphic Compression Minghao Li Ran Ben-Basat S. Vargaftik Chon-In Lao Ke Xu Michael Mitzenmacher Minlan Yu Harvard University 94 19 0 16 Feb 2023
Auto-Parallelizing Large Models with Rhino: A Systematic Approach on Production AI Platform Shiwei Zhang Lansong Diao Siyu Wang Zongyan Cao Yiliang Gu Chang Si Ziji Shi Zhen Zheng Chuan Wu W. Lin AI4CE 54 4 0 16 Feb 2023
Slapo: A Schedule Language for Progressive Optimization of Large Deep Learning Model Training Hongzheng Chen Cody Hao Yu Shuai Zheng Zhen Zhang Zhiru Zhang Yida Wang 84 8 0 16 Feb 2023
Expediting Distributed DNN Training with Device Topology-Aware Graph Deployment Shiwei Zhang Xiaodong Yi Lansong Diao Chuan Wu Siyu Wang W. Lin GNN 41 5 0 13 Feb 2023
Landscape of High-performance Python to Develop Data Science and Machine Learning Applications Oscar Castro P. Bruneau Jean-Sébastien Sottet D. Torregrossa 58 10 0 07 Feb 2023
Colossal-Auto: Unified Automation of Parallelization and Activation Checkpoint for Large-scale Models Yuliang Liu Shenggui Li Jiarui Fang Yan Shao Boyuan Yao Yang You OffRL 83 7 0 06 Feb 2023
LiteVR: Interpretable and Lightweight Cybersickness Detection using Explainable AI Ripan Kumar Kundu Rifatul Islam John Quarles K. A. Hoque 54 10 0 05 Feb 2023
SuperScaler: Supporting Flexible DNN Parallelization via a Unified Abstraction Zhiqi Lin Youshan Miao Guodong Liu Xiaoxiang Shi Quanlu Zhang ... Xu Cao Cheng-Wu Li Mao Yang Lintao Zhang Lidong Zhou 58 6 0 21 Jan 2023
ATP: Adaptive Tensor Parallelism for Foundation Models Shenggan Cheng Ziming Liu Jiangsu Du Yang You 58 6 0 20 Jan 2023
AutoDDL: Automatic Distributed Deep Learning with Near-Optimal Bandwidth Cost Jinfan Chen Shigang Li Ran Guo Jinhui Yuan Torsten Hoefler 57 2 0 17 Jan 2023
Systems for Parallel and Distributed Large-Model Deep Learning Training Kabir Nagrecha GNN VLM MoE 74 7 0 06 Jan 2023
Does compressing activations help model parallel training? S. Bian Dacheng Li Hongyi Wang Eric P. Xing Shivaram Venkataraman 72 9 0 06 Jan 2023
Containerisation for High Performance Computing Systems: Survey and Prospects Naweiluo Zhou Huan Zhou Dennis Hoppe 65 27 0 16 Dec 2022
Multiscale Graph Neural Networks for Protein Residue Contact Map Prediction Kuang Liu R. Kalia Xinlian Liu A. Nakano K. Nomura P. Vashishta R. Zamora-Resendiz 96 2 0 02 Dec 2022
Distributed Deep Reinforcement Learning: A Survey and A Multi-Player Multi-Agent Learning Toolbox Qiyue Yin Tongtong Yu S. Shen Jun Yang Meijing Zhao Kaiqi Huang Bin Liang Liangsheng Wang OffRL 74 13 0 01 Dec 2022
TensAIR: Real-Time Training of Neural Networks from Data-streams Mauro Dalle Lucca Tosi V. Venugopal Martin Theobald 45 1 0 18 Nov 2022
CASPR: Customer Activity Sequence-based Prediction and Representation Pin-Jung Chen Sahil Bhatnagar Sagar Goyal D. Kowalczyk Mayank Shrivastava AI4TS 61 0 0 16 Nov 2022
Aspects of scaling and scalability for flow-based sampling of lattice QCD Ryan Abbott M. S. Albergo Aleksandar Botev D. Boyda Kyle Cranmer ... Ali Razavi Danilo Jimenez Rezende F. Romero-López P. Shanahan Julian M. Urban 116 33 0 14 Nov 2022
On Optimizing the Communication of Model Parallelism Yonghao Zhuang Hexu Zhao Lianmin Zheng Zhuohan Li Eric P. Xing Qirong Ho Joseph E. Gonzalez Ion Stoica Haotong Zhang 111 28 0 10 Nov 2022
Cross-Attention is all you need: Real-Time Streaming Transformers for Personalised Speech Enhancement Shucong Zhang Malcolm Chadwick Alberto Gil C. P. Ramos S. Bhattacharya 57 5 0 08 Nov 2022
A Deep Double Ritz Method (D $^2$ RM) for solving Partial Differential Equations using Neural Networks C. Uriarte David Pardo I. Muga J. Muñoz‐Matute 99 18 0 07 Nov 2022
SOLAR: A Highly Optimized Data Loading Framework for Distributed Training of CNN-based Scientific Surrogates Baixi Sun Xiaodong Yu Chengming Zhang Jiannan Tian Sian Jin K. Iskra Tao Zhou Tekin Bicer Pete Beckman Dingwen Tao 60 1 0 01 Nov 2022
Management of Machine Learning Lifecycle Artifacts: A Survey Marius Schlegel K. Sattler 121 40 0 21 Oct 2022
AMP: Automatically Finding Model Parallel Strategies with Heterogeneity Awareness Dacheng Li Hongyi Wang Eric P. Xing Haotong Zhang MoE 54 24 0 13 Oct 2022
PARAGEN : A Parallel Generation Toolkit Jiangtao Feng Yi Zhou Jun Zhang Xian Qian Liwei Wu Zhexi Zhang Yanming Liu Mingxuan Wang Lei Li Hao Zhou VLM 69 3 0 07 Oct 2022
Cloud Classification with Unsupervised Deep Learning Takuya Kurihana Ian Foster Rebecca Willett S. Jenkins Kathryn Koenig Ruby Werman Ricardo Barros Lourenço Casper Neo Elisabeth Moyer 31 9 0 30 Sep 2022
Optimizing DNN Compilation for Distributed Training with Joint OP and Tensor Fusion Xiaodong Yi Shiwei Zhang Lansong Diao Chuan Wu Zhen Zheng Shiqing Fan Siyu Wang Jun Yang W. Lin 69 4 0 26 Sep 2022
Empirical Analysis on Top-k Gradient Sparsification for Distributed Deep Learning in a Supercomputing Environment Daegun Yoon Sangyoon Oh 85 0 0 18 Sep 2022
A Container-Based Workflow for Distributed Training of Deep Learning Algorithms in HPC Clusters Jose González-Abad Álvaro López García Valentin Kozlov 51 7 0 04 Aug 2022
Neural Nets with a Newton Conjugate Gradient Method on Multiple GPUs Severin Reiz T. Neckel H. Bungartz ODL 65 1 0 03 Aug 2022
Towards Efficient Communications in Federated Learning: A Contemporary Survey Zihao Zhao Yuzhu Mao Yang Liu Linqi Song Ouyang Ye Xinlei Chen Wenbo Ding FedML 112 64 0 02 Aug 2022
Large-scale Knowledge Distillation with Elastic Heterogeneous Computing Resources Ji Liu Daxiang Dong Xi Wang An Qin Xingjian Li P. Valduriez Dejing Dou Dianhai Yu 71 6 0 14 Jul 2022
Identifying Orientation-specific Lipid-protein Fingerprints using Deep Learning Fikret Aydin Konstantia Georgouli Gautham Dharuman J. Glosli F. Lightstone Helgi I. Ingólfsson P. Bremer H. Bhatia 22 0 0 14 Jul 2022
Emerging Patterns in the Continuum Representation of Protein-Lipid Fingerprints Konstantia Georgouli Helgi I. Ingólfsson Fikret Aydin Mark Heimann F. Lightstone P. Bremer H. Bhatia 24 0 0 09 Jul 2022
High-Performance GPU-to-CPU Transpilation and Optimization via High-Level Parallel Constructs William S. Moses Ivan R. Ivanov Jens Domke Toshio Endo J. Doerfert O. Zinenko 23 17 0 01 Jul 2022
Scalable K-FAC Training for Deep Neural Networks with Distributed Preconditioning Lin Zhang Shaoshuai Shi Wei Wang Yue Liu 70 10 0 30 Jun 2022
GMI-DRL: Empowering Multi-GPU Deep Reinforcement Learning with GPU Spatial Multiplexing Yuke Wang Boyuan Feng Ziyi Wang Tong Geng Ang Li Yufei Ding AI4CE 76 0 0 16 Jun 2022
Modern Distributed Data-Parallel Large-Scale Pre-training Strategies For NLP models Haoli Bai MoE 143 5 0 13 Jun 2022
Merak: An Efficient Distributed DNN Training Framework with Automated 3D Parallelism for Giant Foundation Models Zhiquan Lai Shengwei Li Xudong Tang Ke-shi Ge Weijie Liu Yabo Duan Linbo Qiao Dongsheng Li 93 46 0 10 Jun 2022
Tutel: Adaptive Mixture-of-Experts at Scale Changho Hwang Wei Cui Yifan Xiong Ziyue Yang Ze Liu ... Joe Chau Peng Cheng Fan Yang Mao Yang Y. Xiong MoE 205 123 0 07 Jun 2022
Fine-tuning Language Models over Slow Networks using Activation Compression with Guarantees Jue Wang Binhang Yuan Luka Rimanic Yongjun He Tri Dao Beidi Chen Christopher Ré Ce Zhang AI4CE 110 13 0 02 Jun 2022
Decentralized Training of Foundation Models in Heterogeneous Environments Binhang Yuan Yongjun He Jared Davis Tianyi Zhang Tri Dao Beidi Chen Percy Liang Christopher Ré Ce Zhang 125 97 0 02 Jun 2022
Good Intentions: Adaptive Parameter Management via Intent Signaling Alexander Renz-Wieland Andreas Kieslinger R. Gericke Rainer Gemulla Zoi Kaoudi Volker Markl 68 1 0 01 Jun 2022
ByteComp: Revisiting Gradient Compression in Distributed Training Zhuang Wang Yanghua Peng Yibo Zhu T. Ng 66 2 0 28 May 2022
Deep Learning Workload Scheduling in GPU Datacenters: Taxonomy, Challenges and Vision Wei Gao Qi Hu Zhisheng Ye Peng Sun Xiaolin Wang Yingwei Luo Tianwei Zhang Yonggang Wen 137 28 0 24 May 2022