Gradient Diversity: a Key Ingredient for Scalable Distributed Learning

v1v2v3 (latest)

Gradient Diversity: a Key Ingredient for Scalable Distributed Learning

18 June 2017

Max Lam

Dimitris Papailiopoulos

Kannan Ramchandran

Peter L. Bartlett

ArXiv (abs)PDF HTML

Papers citing "Gradient Diversity: a Key Ingredient for Scalable Distributed Learning"

8 / 8 papers shown

Title
Where to Begin? On the Impact of Pre-Training and Initialization in Federated Learning John Nguyen Jianyu Wang Kshitiz Malik Maziar Sanjabi Michael G. Rabbat FedML AI4CE 95 75 0 14 Oct 2022
Federated Learning with Buffered Asynchronous Aggregation John Nguyen Kshitiz Malik Hongyuan Zhan Ashkan Yousefpour Michael G. Rabbat Mani Malek Dzmitry Huba FedML 101 316 0 11 Jun 2021
Accordion: Adaptive Gradient Communication via Critical Learning Regime Identification Saurabh Agarwal Hongyi Wang Kangwook Lee Shivaram Venkataraman Dimitris Papailiopoulos 85 25 0 29 Oct 2020
Improving the convergence of SGD through adaptive batch sizes Scott Sievert Zachary B. Charles ODL 74 8 0 18 Oct 2019
Ray Interference: a Source of Plateaus in Deep Reinforcement Learning Tom Schaul Diana Borsa Joseph Modayil Razvan Pascanu 79 63 0 25 Apr 2019
The Impact of Neural Network Overparameterization on Gradient Confusion and Stochastic Gradient Descent Karthik A. Sankararaman Soham De Zheng Xu Wenjie Huang Tom Goldstein ODL 122 106 0 15 Apr 2019
Slow and Stale Gradients Can Win the Race: Error-Runtime Trade-offs in Distributed SGD Sanghamitra Dutta Gauri Joshi Soumyadip Ghosh Parijat Dube P. Nagpurkar 82 198 0 03 Mar 2018
Stochastic gradient descent performs variational inference, converges to limit cycles for deep networks Pratik Chaudhari Stefano Soatto MLT 106 304 0 30 Oct 2017

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.