On the Outsized Importance of Learning Rates in Local Update Methods

2 July 2020

Papers citing "On the Outsized Importance of Learning Rates in Local Update Methods"

50 / 55 papers shown

Title
Adaptive Client Sampling in Federated Learning via Online Learning with Bandit Feedback Boxin Zhao Lingxiao Wang Mladen Kolar Ziqi Liu Qing Cui Jun Zhou Chaochao Chen FedML 116 11 0 28 Dec 2021
Mime: Mimicking Centralized Stochastic Algorithms in Federated Learning Sai Praneeth Karimireddy Martin Jaggi Satyen Kale M. Mohri Sashank J. Reddi Sebastian U. Stich A. Suresh FedML 136 217 0 08 Aug 2020
Training Keyword Spotting Models on Non-IID Data with Federated Learning Andrew Straiton Hard Kurt Partridge Cameron Nguyen Niranjan A. Subrahmanya Aishanee Shah Pai Zhu Ignacio López Moreno Rajiv Mathews OOD FedML 59 67 0 21 May 2020
FedSplit: An algorithmic framework for fast federated optimization Reese Pathak Martin J. Wainwright FedML 217 184 0 11 May 2020
From Local SGD to Local Fixed-Point Methods for Federated Learning Grigory Malinovsky D. Kovalev Elnur Gasanov Laurent Condat Peter Richtárik FedML 126 117 0 03 Apr 2020
Adaptive Federated Optimization Sashank J. Reddi Zachary B. Charles Manzil Zaheer Zachary Garrett Keith Rush Jakub Konecný Sanjiv Kumar H. B. McMahan FedML 183 1,452 0 29 Feb 2020
Personalized Federated Learning: A Meta-Learning Approach Alireza Fallah Aryan Mokhtari Asuman Ozdaglar FedML 165 572 0 19 Feb 2020
Is Local SGD Better than Minibatch SGD? Blake E. Woodworth Kumar Kshitij Patel Sebastian U. Stich Zhen Dai Brian Bullins H. B. McMahan Ohad Shamir Nathan Srebro FedML 74 254 0 18 Feb 2020
Advances and Open Problems in Federated Learning Peter Kairouz H. B. McMahan Brendan Avent A. Bellet M. Bennis ... Zheng Xu Qiang Yang Felix X. Yu Han Yu Sen Zhao FedML AI4CE 270 6,285 0 10 Dec 2019
Local AdaAlter: Communication-Efficient Stochastic Gradient Descent with Adaptive Learning Rates Cong Xie Oluwasanmi Koyejo Indranil Gupta Yanghua Peng 64 42 0 20 Nov 2019
Can You Really Backdoor Federated Learning? Ziteng Sun Peter Kairouz A. Suresh H. B. McMahan FedML 75 578 0 18 Nov 2019
Generative Models for Effective ML on Private, Decentralized Datasets S. Augenstein H. B. McMahan Daniel Ramage Swaroop Indra Ramaswamy Peter Kairouz Mingqing Chen Rajiv Mathews Blaise Agüera y Arcas SyDa 87 183 0 15 Nov 2019
SCAFFOLD: Stochastic Controlled Averaging for Federated Learning Sai Praneeth Karimireddy Satyen Kale M. Mohri Sashank J. Reddi Sebastian U. Stich A. Suresh FedML 67 347 0 14 Oct 2019
Federated Learning of N-gram Language Models Mingqing Chen A. Suresh Rajiv Mathews Adeline Wong Cyril Allauzen F. Beaufays Michael Riley FedML 111 75 0 08 Oct 2019
The Non-IID Data Quagmire of Decentralized Machine Learning Kevin Hsieh Amar Phanishayee O. Mutlu Phillip B. Gibbons 167 573 0 01 Oct 2019
FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization Amirhossein Reisizadeh Aryan Mokhtari Hamed Hassani Ali Jadbabaie Ramtin Pedarsani FedML 260 774 0 28 Sep 2019
Improving Federated Learning Personalization via Model Agnostic Meta Learning Yihan Jiang Jakub Konecný Keith Rush Sreeram Kannan FedML 80 602 0 27 Sep 2019
Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML Aniruddh Raghu M. Raghu Samy Bengio Oriol Vinyals 305 647 0 19 Sep 2019
Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification T. Hsu Qi Matthew Brown FedML 145 1,164 0 13 Sep 2019
Tighter Theory for Local SGD on Identical and Heterogeneous Data Ahmed Khaled Konstantin Mishchenko Peter Richtárik 76 435 0 10 Sep 2019
Meta-Learning with Implicit Gradients Aravind Rajeswaran Chelsea Finn Sham Kakade Sergey Levine 116 858 0 10 Sep 2019
On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithms Alireza Fallah Aryan Mokhtari Asuman Ozdaglar 88 225 0 27 Aug 2019
Federated Learning: Challenges, Methods, and Future Directions Tian Li Anit Kumar Sahu Ameet Talwalkar Virginia Smith FedML 132 4,543 0 21 Aug 2019
Lookahead Optimizer: k steps forward, 1 step back Michael Ruogu Zhang James Lucas Geoffrey E. Hinton Jimmy Ba ODL 157 732 0 19 Jul 2019
On the Convergence of FedAvg on Non-IID Data Xiang Li Kaixuan Huang Wenhao Yang Shusen Wang Zhihua Zhang FedML 166 2,352 0 04 Jul 2019
Robust Federated Learning in a Heterogeneous Environment Avishek Ghosh Justin Hong Dong Yin Kannan Ramchandran OOD FedML 63 219 0 16 Jun 2019
Adaptive Gradient-Based Meta-Learning Methods M. Khodak Maria-Florina Balcan Ameet Talwalkar FedML 95 356 0 06 Jun 2019
Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computations Debraj Basu Deepesh Data C. Karakuş Suhas Diggavi MQ 63 406 0 06 Jun 2019
Fair Resource Allocation in Federated Learning Tian Li Maziar Sanjabi Ahmad Beirami Virginia Smith FedML 190 804 0 25 May 2019
Robust and Communication-Efficient Federated Learning from Non-IID Data Felix Sattler Simon Wiedemann K. Müller Wojciech Samek FedML 74 1,362 0 07 Mar 2019
Federated Machine Learning: Concept and Applications Qiang Yang Yang Liu Tianjian Chen Yongxin Tong FedML 78 2,322 0 13 Feb 2019
Towards Federated Learning at Scale: System Design Keith Bonawitz Hubert Eichner W. Grieskamp Dzmitry Huba A. Ingerman ... H. B. McMahan Timon Van Overveldt David Petrou Daniel Ramage Jason Roselander FedML 126 2,674 0 04 Feb 2019
Agnostic Federated Learning M. Mohri Gary Sivek A. Suresh FedML 136 937 0 01 Feb 2019
Federated Optimization in Heterogeneous Networks Tian Li Anit Kumar Sahu Manzil Zaheer Maziar Sanjabi Ameet Talwalkar Virginia Smith FedML 190 5,227 0 14 Dec 2018
Applied Federated Learning: Improving Google Keyboard Query Suggestions Timothy Yang Galen Andrew Hubert Eichner Haicheng Sun Wei Li Nicholas Kong Daniel Ramage F. Beaufays FedML 87 626 0 07 Dec 2018
LEAF: A Benchmark for Federated Settings S. Caldas Sai Meher Karthik Duddu Peter Wu Tian Li Jakub Konecný H. B. McMahan Virginia Smith Ameet Talwalkar FedML 158 1,423 0 03 Dec 2018
Federated Learning for Mobile Keyboard Prediction Andrew Straiton Hard Kanishka Rao Zhifeng Lin Swaroop Indra Ramaswamy Youjie Li S. Augenstein Alex Schwing M. Annavaram A. Avestimehr FedML 136 1,546 0 08 Nov 2018
How to train your MAML Antreas Antoniou Harrison Edwards Amos Storkey 74 777 0 22 Oct 2018
Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithms Jianyu Wang Gauri Joshi 172 349 0 22 Aug 2018
Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learning Hao Yu Sen Yang Shenghuo Zhu MoMe FedML 79 609 0 17 Jul 2018
Meta-Learning with Latent Embedding Optimization Andrei A. Rusu Dushyant Rao Jakub Sygnowski Oriol Vinyals Razvan Pascanu Simon Osindero R. Hadsell 148 1,372 0 16 Jul 2018
How To Backdoor Federated Learning Eugene Bagdasaryan Andreas Veit Yiqing Hua D. Estrin Vitaly Shmatikov SILM FedML 97 1,929 0 02 Jul 2018
Local SGD Converges Fast and Communicates Little Sebastian U. Stich FedML 187 1,070 0 24 May 2018
Federated Learning for Ultra-Reliable Low-Latency V2V Communications S. Samarakoon M. Bennis Walid Saad Merouane Debbah 65 228 0 11 May 2018
Group Normalization Yuxin Wu Kaiming He 239 3,672 0 22 Mar 2018
On First-Order Meta-Learning Algorithms Alex Nichol Joshua Achiam John Schulman 237 2,238 0 08 Mar 2018
Recasting Gradient-Based Meta-Learning as Hierarchical Bayes Erin Grant Chelsea Finn Sergey Levine Trevor Darrell Thomas Griffiths BDL 90 510 0 26 Jan 2018
Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour Priya Goyal Piotr Dollár Ross B. Girshick P. Noordhuis Lukasz Wesolowski Aapo Kyrola Andrew Tulloch Yangqing Jia Kaiming He 3DH 128 3,685 0 08 Jun 2017
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks Chelsea Finn Pieter Abbeel Sergey Levine OOD 829 11,952 0 09 Mar 2017
Federated Learning: Strategies for Improving Communication Efficiency Jakub Konecný H. B. McMahan Felix X. Yu Peter Richtárik A. Suresh Dave Bacon FedML 312 4,657 0 18 Oct 2016