v1v2v3 (latest)

Horovod: fast and easy distributed deep learning in TensorFlow

15 February 2018

Papers citing "Horovod: fast and easy distributed deep learning in TensorFlow"

50 / 454 papers shown

Title
Revisiting Pre-trained Language Models and their Evaluation for Arabic Natural Language Understanding Abbas Ghaddar Yimeng Wu Sunyam Bagga Ahmad Rashid Khalil Bibi ... Zhefeng Wang Baoxing Huai Xin Jiang Qun Liu Philippe Langlais 65 6 0 21 May 2022
Scalable algorithms for physics-informed neural and graph networks K. Shukla Mengjia Xu N. Trask George Karniadakis PINN AI4CE 131 41 0 16 May 2022
DL4DS -- Deep Learning for empirical DownScaling C. G. Gomez Gonzalez AI4Cl AI4CE 46 6 0 07 May 2022
dPRO: A Generic Profiling and Optimization System for Expediting Distributed DNN Training Han Hu Chenyu Jiang Yuchen Zhong Size Zheng Chuan Wu Yibo Zhu Yanghua Peng Chuanxiong Guo 52 19 0 05 May 2022
SMLT: A Serverless Framework for Scalable and Adaptive Machine Learning Design and Training Ahsan Ali Syed Zawad Paarijaat Aditya Istemi Ekin Akkus Ruichuan Chen Feng Yan 53 9 0 04 May 2022
MiCS: Near-linear Scaling for Training Gigantic Model on Public Cloud Zhen Zhang Shuai Zheng Yida Wang Justin Chiu George Karypis Trishul Chilimbi Mu Li Xin Jin 67 39 0 30 Apr 2022
Distributed intelligence on the Edge-to-Cloud Continuum: A systematic literature review Daniel Rosendo Alexandru Costan P. Valduriez Gabriel Antoniu 62 86 0 29 Apr 2022
Bamboo: Making Preemptible Instances Resilient for Affordable Training of Large DNNs John Thorpe Pengzhan Zhao Jon Eyolfson Yifan Qiao Zhihao Jia Minjia Zhang Ravi Netravali Guoqing Harry Xu 83 58 0 26 Apr 2022
Analysing the Influence of Attack Configurations on the Reconstruction of Medical Images in Federated Learning M. Dahlgaard Morten Wehlast Jorgensen N. Fuglsang Hiba Nassar FedML AAML 72 2 0 25 Apr 2022
Data Debugging with Shapley Importance over End-to-End Machine Learning Pipelines Bojan Karlavs David Dao Matteo Interlandi Yue Liu Sebastian Schelter Wentao Wu Ce Zhang TDI 67 27 0 23 Apr 2022
FPGA-based AI Smart NICs for Scalable Distributed AI Training Systems Rui Ma E. Georganas A. Heinecke Andrew Boutros Eriko Nurvitadhi GNN 52 13 0 22 Apr 2022
Efficient Pipeline Planning for Expedited Distributed DNN Training Ziyue Luo Xiaodong Yi Guoping Long Shiqing Fan Chuan Wu Jun Yang Wei Lin 67 16 0 22 Apr 2022
Sign Bit is Enough: A Learning Synchronization Framework for Multi-hop All-reduce with Ultimate Compression Feijie Wu Shiqi He Song Guo Zhihao Qu Yining Qi W. Zhuang Jie Zhang 67 9 0 14 Apr 2022
PICASSO: Unleashing the Potential of GPU-centric Training for Wide-and-deep Recommender Systems Yuanxing Zhang Langshi Chen Siran Yang Man Yuan Hui-juan Yi ... Yong Li Dingyang Zhang Wei Lin Lin Qu Bo Zheng 85 32 0 11 Apr 2022
Elastic Model Aggregation with Parameter Service Juncheng Gu Mosharaf Chowdhury Kang G. Shin Aditya Akella 28 4 0 07 Apr 2022
PerfectDou: Dominating DouDizhu with Perfect Information Distillation Yang Guan Minghuan Liu Weijun Hong Weinan Zhang Fei Fang Guangjun Zeng Yue Lin 119 28 0 30 Mar 2022
FLUTE: A Scalable, Extensible Framework for High-Performance Federated Learning Simulations Mirian Hipolito Garcia Andre Manoel Daniel Madrigal Diaz Fatemehsadat Mireshghallah Robert Sim Dimitrios Dimitriadis FedML 92 57 0 25 Mar 2022
Locally Asynchronous Stochastic Gradient Descent for Decentralised Deep Learning Tomer Avidor Nadav Tal-Israel 28 2 0 24 Mar 2022
The Tiny-Tasks Granularity Trade-Off: Balancing overhead vs. performance in parallel systems Stefan Bora Brenton D. Walker Markus Fidler 21 1 0 23 Feb 2022
A Deep Learning Model for Forecasting Global Monthly Mean Sea Surface Temperature Anomalies John Taylor Ming Feng AI4Cl 35 32 0 21 Feb 2022
Where Is My Training Bottleneck? Hidden Trade-Offs in Deep Learning Preprocessing Pipelines Alexander Isenko R. Mayer Jeffrey Jedele Hans-Arno Jacobsen 104 26 0 17 Feb 2022
Benchmark Assessment for DeepSpeed Optimization Library G. Liang I. Alsmadi 59 3 0 12 Feb 2022
Applications of Machine Learning to Lattice Quantum Field Theory D. Boyda Salvatore Cali Sam Foreman L. Funcke D. Hackett ... Gert Aarts A. Alexandru Xiao-Yong Jin B. Lucini P. Shanahan AI4CE 74 19 0 10 Feb 2022
Efficient Direct-Connect Topologies for Collective Communications Liangyu Zhao Siddharth Pal Tapan Chugh Weiyang Wang Jason Fantl P. Basu J. Khoury Arvind Krishnamurthy 99 7 0 07 Feb 2022
Distributed Learning With Sparsified Gradient Differences Yicheng Chen Rick S. Blum Martin Takáč Brian M. Sadler 77 15 0 05 Feb 2022
GADGET: Online Resource Optimization for Scheduling Ring-All-Reduce Learning Jobs Menglu Yu Ye Tian Bo Ji Chuan Wu Hridesh Rajan Jia-Wei Liu 39 19 0 02 Feb 2022
TopoOpt: Co-optimizing Network Topology and Parallelization Strategy for Distributed Training Jobs Weiyang Wang Moein Khazraee Zhizhen Zhong M. Ghobadi Zhihao Jia Dheevatsa Mudigere Ying Zhang A. Kewitsch 131 93 0 01 Feb 2022
You May Not Need Ratio Clipping in PPO Mingfei Sun Vitaly Kurin Guoqing Liu Sam Devlin Tao Qin Katja Hofmann Shimon Whiteson 62 16 0 31 Jan 2022
Distributed SLIDE: Enabling Training Large Neural Networks on Low Bandwidth and Simple CPU-Clusters via Model Parallelism and Sparsity Minghao Yan Nicholas Meisburger Tharun Medini Anshumali Shrivastava 86 6 0 29 Jan 2022
Benchmarking Resource Usage for Efficient Distributed Deep Learning Nathan C. Frey Baolin Li Joseph McDonald Dan Zhao Michael Jones David Bestor Devesh Tiwari V. Gadepally S. Samsi 92 9 0 28 Jan 2022
Alpa: Automating Inter- and Intra-Operator Parallelism for Distributed Deep Learning Lianmin Zheng Zhuohan Li Hao Zhang Yonghao Zhuang Zhifeng Chen ... Yuanzhong Xu Danyang Zhuo Eric P. Xing Joseph E. Gonzalez Ion Stoica MoE 141 104 0 28 Jan 2022
GC3: An Optimizing Compiler for GPU Collective Communication M. Cowan Saeed Maleki Madan Musuvathi Olli Saarikivi Yifan Xiong GNN 80 11 0 27 Jan 2022
AI Technical Considerations: Data Storage, Cloud usage and AI Pipeline P. V. van Ooijen Erfan Darzidehkalani Andre Dekker 22 4 0 20 Jan 2022
Near-Optimal Sparse Allreduce for Distributed Deep Learning Shigang Li Torsten Hoefler 62 53 0 19 Jan 2022
Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next S. Cuomo Vincenzo Schiano Di Cola F. Giampaolo G. Rozza Maizar Raissi F. Piccialli PINN 138 1,302 0 14 Jan 2022
SplitBrain: Hybrid Data and Model Parallel Deep Learning Farley Lai Asim Kadav E. Kruus FedML MoE 10 2 0 31 Dec 2021
FRuDA: Framework for Distributed Adversarial Domain Adaptation Shaoduo Gan Akhil Mathur Anton Isopoussu F. Kawsar N. Bianchi-Berthouze Nicholas D. Lane 70 13 0 26 Dec 2021
ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation Shuohuan Wang Yu Sun Yang Xiang Zhihua Wu Siyu Ding ... Tian Wu Wei Zeng Ge Li Wen Gao Haifeng Wang ELM 92 78 0 23 Dec 2021
HET: Scaling out Huge Embedding Model Training via Cache-enabled Distributed Framework Xupeng Miao Hailin Zhang Yining Shi Xiaonan Nie Zhi-Xin Yang Yangyu Tao Tengjiao Wang 67 58 0 14 Dec 2021
JABER and SABER: Junior and Senior Arabic BERt Abbas Ghaddar Yimeng Wu Ahmad Rashid Khalil Bibi Mehdi Rezagholizadeh ... Zhefeng Wang Baoxing Huai Xin Jiang Qun Liu Philippe Langlais 55 5 0 08 Dec 2021
End-to-end Adaptive Distributed Training on PaddlePaddle Yulong Ao Zhihua Wu Dianhai Yu Weibao Gong Zhiqing Kui ... Yanjun Ma Tian Wu Haifeng Wang Wei Zeng Chao Yang 118 11 0 06 Dec 2021
COMET: A Novel Memory-Efficient Deep Learning Training Framework by Using Error-Bounded Lossy Compression Sian Jin Chengming Zhang Xintong Jiang Yunhe Feng Hui Guan Guanpeng Li Shuaiwen Leon Song Dingwen Tao 46 25 0 18 Nov 2021
Deep neural networks-based denoising models for CT imaging and their efficacy Prabhat Kc R. Zeng M. M. Farhangi Kyle J. Myers 29 20 0 18 Nov 2021
Benchmarking and scaling of deep learning models for land cover image classification Ioannis Papoutsis Nikolaos Ioannis Bountos Angelos Zavras Dimitrios Michail Christos Tryfonopoulos 99 61 0 18 Nov 2021
CGX: Adaptive System Support for Communication-Efficient Deep Learning I. Markov Hamidreza Ramezanikebrya Dan Alistarh GNN 82 5 0 16 Nov 2021
HADFL: Heterogeneity-aware Decentralized Federated Learning Framework Jing Cao Zirui Lian Weihong Liu Zongwei Zhu Cheng Ji FedML 41 18 0 16 Nov 2021
Task allocation for decentralized training in heterogeneous environment Yongyue Chao Ming-Ray Liao Jiaxin Gao 26 0 0 16 Nov 2021
AI in Human-computer Gaming: Techniques, Challenges and Opportunities Qiyue Yin Jun Yang Kaiqi Huang Meijing Zhao Wancheng Ni Bin Liang Yan Huang Shu Wu Liangsheng Wang 54 21 0 15 Nov 2021
Persia: An Open, Hybrid System Scaling Deep Learning-based Recommenders up to 100 Trillion Parameters Xiangru Lian Binhang Yuan Xuefeng Zhu Yulong Wang Yongjun He ... Lei Yuan Hai-bo Yu Sen Yang Ce Zhang Ji Liu VLM 99 36 0 10 Nov 2021
A Survey and Empirical Evaluation of Parallel Deep Learning Frameworks Daniel Nichols Siddharth Singh Shuqing Lin A. Bhatele OOD 57 9 0 09 Nov 2021