AxoNN: An asynchronous, message-driven parallel framework for extreme-scale deep learning

25 October 2021

Papers citing "AxoNN: An asynchronous, message-driven parallel framework for extreme-scale deep learning"

4 / 4 papers shown

Title
Democratizing AI: Open-source Scalable LLM Training on GPU-based Supercomputers Siddharth Singh Prajwal Singhania Aditya K. Ranjan John Kirchenbauer Jonas Geiping ... Abhimanyu Hans Manli Shu Aditya Tomar Tom Goldstein A. Bhatele 102 2 0 12 Feb 2025
Loki: Low-Rank Keys for Efficient Sparse Attention Prajwal Singhania Siddharth Singh Shwai He S. Feizi A. Bhatele 32 13 0 04 Jun 2024
ZeRO-Offload: Democratizing Billion-Scale Model Training Jie Ren Samyam Rajbhandari Reza Yazdani Aminabadi Olatunji Ruwase Shuangyang Yang Minjia Zhang Dong Li Yuxiong He MoE 177 414 0 18 Jan 2021
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 245 1,821 0 17 Sep 2019