Proteus: Simulating the Performance of Distributed DNN Training

Proteus: Simulating the Performance of Distributed DNN Training

4 June 2023

Xingcheng Zhang

Papers citing "Proteus: Simulating the Performance of Distributed DNN Training"

8 / 8 papers shown

Title
PipeWeaver: Addressing Data Dynamicity in Large Multimodal Model Training with Dynamic Interleaved Pipeline Zhenliang Xue Hanpeng Hu Xing Chen Yimin Jiang Yixin Song Zeyu Mi Yibo Zhu Daxin Jiang Yubin Xia Haibo Chen 49 0 0 19 Apr 2025
Maya: Optimizing Deep Learning Training Workloads using Emulated Virtual Accelerators Srihas Yarlagadda A. Agrawal Elton Pinto Hakesh Darapaneni Mitali Meratwal Shivam Mittal Pranavi Bajjuri Shri Kiran Srinivasan Alexey Tumanov 83 0 0 26 Mar 2025
Mist: Efficient Distributed Training of Large Language Models via Memory-Parallelism Co-Optimization Zhanda Zhu Christina Giannoula Muralidhar Andoorveedu Qidong Su Karttikeya Mangalam Bojian Zheng Gennady Pekhimenko VLM MoE 59 0 0 24 Mar 2025
Echo: Simulating Distributed Training At Scale Yicheng Feng Yuetao Chen Kaiwen Chen Jingzong Li Tianyuan Wu Peng Cheng Chuan Wu Wei Wang Tsung-Yi Ho Hong Xu 84 2 0 17 Dec 2024
Towards a Standardized Representation for Deep Learning Collective Algorithms Jinsun Yoo William Won Meghan Cowan Nan Jiang Benjamin Klenk Srinivas Sridharan Tushar Krishna 34 1 0 20 Aug 2024
Vidur: A Large-Scale Simulation Framework For LLM Inference Amey Agrawal Nitin Kedia Jayashree Mohan Ashish Panwar Nipun Kwatra Bhargav S. Gulavani Ramachandran Ramjee Alexey Tumanov VLM 40 38 0 08 May 2024
ZeRO-Offload: Democratizing Billion-Scale Model Training Jie Ren Samyam Rajbhandari Reza Yazdani Aminabadi Olatunji Ruwase Shuangyang Yang Minjia Zhang Dong Li Yuxiong He MoE 177 417 0 18 Jan 2021
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism Mohammad Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 245 1,836 0 17 Sep 2019