On Optimizing the Communication of Model Parallelism

10 November 2022

Papers citing "On Optimizing the Communication of Model Parallelism"

3 / 3 papers shown

Title
Hybrid-Parallel: Achieving High Performance and Energy Efficient Distributed Inference on Robots Zekai Sun Xiuxian Guan Junming Wang Haoze Song Yuhao Qing Tianxiang Shen Dong Huang Fangming Liu Heming Cui 34 0 0 29 May 2024
Automated Tensor Model Parallelism with Overlapped Communication for Efficient Foundation Model Training Shengwei Li Zhiquan Lai Yanqi Hao Weijie Liu Ke-shi Ge Xiaoge Deng Dongsheng Li KaiCheng Lu 16 10 0 25 May 2023
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 245 1,826 0 17 Sep 2019