Layer-Parallel Training with GPU Concurrency of Deep Residual Neural
Networks via Nonlinear Multigrid

Layer-Parallel Training with GPU Concurrency of Deep Residual Neural Networks via Nonlinear Multigrid

14 July 2020

Michael Jones

Albert Reuther

Papers citing "Layer-Parallel Training with GPU Concurrency of Deep Residual Neural Networks via Nonlinear Multigrid"

10 / 10 papers shown

Title
DeepPCR: Parallelizing Sequential Operations in Neural Networks Federico Danieli Miguel Sarabia Xavier Suau Yuan-Sen Ting Luca Zappella 21 1 0 28 Sep 2023
Parareal with a physics-informed neural network as coarse propagator A. Ibrahim Sebastian Götschel Daniel Ruprecht 31 8 0 07 Mar 2023
Multilevel-in-Layer Training for Deep Neural Network Regression Colin Ponce Ruipeng Li Christina Mao P. Vassilevski AI4CE 19 1 0 11 Nov 2022
Parallel Training of GRU Networks with a Multi-Grid Solver for Long Sequences G. Moon E. Cyr 25 5 0 07 Mar 2022
Layer-Parallel Training of Residual Networks with Auxiliary-Variable Networks Qi Sun Hexin Dong Zewei Chen Jiacheng Sun Zhenguo Li Bin Dong 27 1 0 10 Dec 2021
Globally Convergent Multilevel Training of Deep Residual Networks Alena Kopanicáková Rolf Krause 37 15 0 15 Jul 2021
ResIST: Layer-Wise Decomposition of ResNets for Distributed Training Chen Dun Cameron R. Wolfe C. Jermaine Anastasios Kyrillidis 16 21 0 02 Jul 2021
Differentiable Multiple Shooting Layers Stefano Massaroli Michael Poli Sho Sonoda Taji Suzuki Jinkyoo Park Atsushi Yamashita Hajime Asama AI4CE 11 18 0 07 Jun 2021
GIST: Distributed Training for Large-Scale Graph Convolutional Networks Cameron R. Wolfe Jingkang Yang Arindam Chowdhury Chen Dun Artun Bayer Santiago Segarra Anastasios Kyrillidis BDL GNN LRM 49 9 0 20 Feb 2021
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 245 1,826 0 17 Sep 2019