MuLoCo: Muon is a practical inner optimizer for DiLoCo

29 May 2025

Papers citing "MuLoCo: Muon is a practical inner optimizer for DiLoCo"

2 / 2 papers shown

Title
Communication-Efficient Language Model Training Scales Reliably and Robustly: Scaling Laws for DiLoCo Zachary B. Charles Gabriel Teston Lucio Dery Keith Rush Nova Fallen Zachary Garrett Arthur Szlam Arthur Douillard 461 6 0 12 Mar 2025
Streaming DiLoCo with overlapping communication: Towards a Distributed Free Lunch Arthur Douillard Yanislav Donchev Keith Rush Satyen Kale Zachary Charles ... Jiajun Shen Alexandre Ramé Arthur Szlam MarcÁurelio Ranzato P. Barham 138 8 0 30 Jan 2025