GNNPipe: Scaling Deep GNN Training with Pipelined Model Parallelism

19 August 2023

Papers citing "GNNPipe: Scaling Deep GNN Training with Pipelined Model Parallelism"

33 / 33 papers shown

Title
GSplit: Scaling Graph Neural Network Training on Large Graphs via Split-Parallelism Sandeep Polisetty Juelin Liu Kobi Falus Yi R. Fung Seung-Hwan Lim Hui Guan Marco Serafini GNN 73 11 0 24 Mar 2023
Distributed Graph Neural Network Training: A Survey Yingxia Shao Hongzheng Li Xizhi Gu Hongbo Yin Yawen Li Xupeng Miao Wentao Zhang Tengjiao Wang Lei Chen GNN AI4CE 80 60 0 01 Nov 2022
MGG: Accelerating Graph Neural Networks with Fine-grained intra-kernel Communication-Computation Pipelining on Multi-GPU Platforms Yuke Wang Boyuan Feng Zheng Wang Tong Geng Kevin J. Barker Ang Li Yufei Ding GNN 57 27 0 14 Sep 2022
BNS-GCN: Efficient Full-Graph Training of Graph Convolutional Networks with Partition-Parallelism and Random Boundary Node Sampling Cheng Wan Youjie Li Ang Li Namjae Kim Yingyan Lin GNN 79 76 0 21 Mar 2022
PipeGCN: Efficient Full-Graph Training of Graph Convolutional Networks with Pipelined Feature Communication Cheng Wan Youjie Li Cameron R. Wolfe Anastasios Kyrillidis Namjae Kim Yingyan Lin GNN 58 70 0 20 Mar 2022
Decoupling the Depth and Scope of Graph Neural Networks Hanqing Zeng Muhan Zhang Yinglong Xia Ajitesh Srivastava Andrey Malevich Rajgopal Kannan Viktor Prasanna Long Jin Ren Chen GNN 80 145 0 19 Jan 2022
Training Graph Neural Networks with 1000 Layers Guohao Li Matthias Muller Guohao Li V. Koltun GNN AI4CE 81 240 0 14 Jun 2021
GNNAutoScale: Scalable and Expressive Graph Neural Networks via Historical Embeddings Matthias Fey J. E. Lenssen F. Weichert J. Leskovec GNN 38 134 0 10 Jun 2021
DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphs Da Zheng Chao Ma Minjie Wang Jinjing Zhou Qidong Su Xiang Song Quan Gan Zheng Zhang George Karypis FedML GNN 50 248 0 11 Oct 2020
Neural Subgraph Matching Rex Ying Ying Zhaoyu Lou Jiaxuan You Chengtao Wen A. Canedo J. Leskovec GNN 41 93 0 06 Jul 2020
Simple and Deep Graph Convolutional Networks Ming Chen Zhewei Wei Zengfeng Huang Bolin Ding Yaliang Li GNN 116 1,485 0 04 Jul 2020
DAPPLE: A Pipelined Data Parallel Approach for Training Large Models Shiqing Fan Yi Rong Chen Meng Zongyan Cao Siyu Wang ... Jun Yang Lixue Xia Lansong Diao Xiaoyong Liu Wei Lin 82 237 0 02 Jul 2020
DeeperGCN: All You Need to Train Deeper GCNs Guohao Li Chenxin Xiong Ali K. Thabet Guohao Li GNN 183 442 0 13 Jun 2020
Towards Deeper Graph Neural Networks with Differentiable Group Normalization Kaixiong Zhou Xiao Huang Yuening Li Daochen Zha Rui Chen Xia Hu 120 205 0 12 Jun 2020
Bayesian Graph Neural Networks with Adaptive Connection Sampling Arman Hasanzadeh Ehsan Hajiramezanali Shahin Boluki Mingyuan Zhou N. Duffield Krishna R. Narayanan Xiaoning Qian BDL 46 118 0 07 Jun 2020
Layer-Dependent Importance Sampling for Training Deep and Large Graph Convolutional Networks Difan Zou Ziniu Hu Yewen Wang Song Jiang Yizhou Sun Quanquan Gu GNN 90 282 0 17 Nov 2019
Multi-scale Attributed Node Embedding Benedek Rozemberczki Carl Allen Rik Sarkar GNN 252 860 0 28 Sep 2019
DropEdge: Towards Deep Graph Convolutional Networks on Node Classification Yu Rong Wenbing Huang Tingyang Xu Junzhou Huang 104 1,339 0 25 Jul 2019
GraphSAINT: Graph Sampling Based Inductive Learning Method Hanqing Zeng Hongkuan Zhou Ajitesh Srivastava Rajgopal Kannan Viktor Prasanna GNN 137 966 0 10 Jul 2019
Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks Wei-Lin Chiang Xuanqing Liu Si Si Yang Li Samy Bengio Cho-Jui Hsieh GNN 142 1,272 0 20 May 2019
Fast Graph Representation Learning with PyTorch Geometric Matthias Fey J. E. Lenssen 3DH GNN 3DPC 218 4,336 0 06 Mar 2019
How Powerful are Graph Neural Networks? Keyulu Xu Weihua Hu J. Leskovec Stefanie Jegelka GNN 232 7,638 0 01 Oct 2018
FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling Jie Chen Tengfei Ma Cao Xiao GNN 141 1,513 0 30 Jan 2018
Dynamic Graph CNN for Learning on Point Clouds Yue Wang Yongbin Sun Ziwei Liu Sanjay E. Sarma M. Bronstein Justin Solomon GNN 3DPC 255 6,132 0 24 Jan 2018
Graph Attention Networks Petar Velickovic Guillem Cucurull Arantxa Casanova Adriana Romero Pietro Lio Yoshua Bengio GNN 454 20,124 0 30 Oct 2017
Inductive Representation Learning on Large Graphs William L. Hamilton Z. Ying J. Leskovec 472 15,218 0 07 Jun 2017
Neural Message Passing for Quantum Chemistry Justin Gilmer S. Schoenholz Patrick F. Riley Oriol Vinyals George E. Dahl 580 7,441 0 04 Apr 2017
Semi-Supervised Classification with Graph Convolutional Networks Thomas Kipf Max Welling GNN SSL 597 28,999 0 09 Sep 2016
Densely Connected Convolutional Networks Gao Huang Zhuang Liu Laurens van der Maaten Kilian Q. Weinberger PINN 3DV 758 36,781 0 25 Aug 2016
Layer Normalization Jimmy Lei Ba J. Kiros Geoffrey E. Hinton 381 10,481 0 21 Jul 2016
Rethinking the Inception Architecture for Computer Vision Christian Szegedy Vincent Vanhoucke Sergey Ioffe Jonathon Shlens Z. Wojna 3DV BDL 872 27,350 0 02 Dec 2015
Going Deeper with Convolutions Christian Szegedy Wei Liu Yangqing Jia P. Sermanet Scott E. Reed Dragomir Anguelov D. Erhan Vincent Vanhoucke Andrew Rabinovich 440 43,635 0 17 Sep 2014
One weird trick for parallelizing convolutional neural networks A. Krizhevsky GNN 88 1,302 0 23 Apr 2014