Optimal Distributed Online Prediction using Mini-Batches

7 December 2010

Papers citing "Optimal Distributed Online Prediction using Mini-Batches"

50 / 97 papers shown

Title
Weight for Robustness: A Comprehensive Approach towards Optimal Fault-Tolerant Asynchronous ML T. Dahan Kfir Y. Levy 65 0 0 17 Jan 2025
Revisiting LocalSGD and SCAFFOLD: Improved Rates and Missing Analysis Ruichen Luo Sebastian U Stich Samuel Horváth Martin Takáč 38 0 0 08 Jan 2025
Faster Stochastic Optimization with Arbitrary Delays via Asynchronous Mini-Batching Amit Attia Ofir Gaash Tomer Koren 40 0 0 14 Aug 2024
The Limits and Potentials of Local SGD for Distributed Heterogeneous Learning with Intermittent Communication Kumar Kshitij Patel Margalit Glasgow Ali Zindari Lingxiao Wang Sebastian U. Stich Ziheng Cheng Nirmit Joshi Nathan Srebro 48 6 0 19 May 2024
Stochastic Online Optimization for Cyber-Physical and Robotic Systems Hao Ma M. Zeilinger Michael Muehlebach 44 0 0 08 Apr 2024
On the Effect of Defections in Federated Learning and How to Prevent Them Minbiao Han Kumar Kshitij Patel Han Shao Lingxiao Wang FedML 33 3 0 28 Nov 2023
Federated Multi-Level Optimization over Decentralized Networks Shuoguang Yang Xuezhou Zhang Mengdi Wang AI4CE 28 0 0 10 Oct 2023
DropCompute: simple and more robust distributed synchronous training via compute variance reduction Niv Giladi Shahar Gottlieb Moran Shkolnik A. Karnieli Ron Banner Elad Hoffer Kfir Y. Levy Daniel Soudry 33 2 0 18 Jun 2023
Repeated Random Sampling for Minimizing the Time-to-Accuracy of Learning Patrik Okanovic R. Waleffe Vasilis Mageirakos Konstantinos E. Nikolakakis Amin Karbasi Dionysis Kalogerias Nezihe Merve Gürel Theodoros Rekatsinas DD 45 12 0 28 May 2023
SLowcal-SGD: Slow Query Points Improve Local-SGD for Stochastic Convex Optimization Kfir Y. Levy Kfir Y. Levy FedML 43 2 0 09 Apr 2023
Two Losses Are Better Than One: Faster Optimization Using a Cheaper Proxy Blake E. Woodworth Konstantin Mishchenko Francis R. Bach 42 6 0 07 Feb 2023
ReSQueing Parallel and Private Stochastic Convex Optimization Y. Carmon A. Jambulapati Yujia Jin Y. Lee Daogao Liu Aaron Sidford Kevin Tian FedML 22 12 0 01 Jan 2023
RCD-SGD: Resource-Constrained Distributed SGD in Heterogeneous Environment via Submodular Partitioning Haoze He Parijat Dube 15 1 0 02 Nov 2022
NET-FLEET: Achieving Linear Convergence Speedup for Fully Decentralized Federated Learning with Heterogeneous Data Xin Zhang Minghong Fang Zhuqing Liu Haibo Yang Jia-Wei Liu Zhengyuan Zhu FedML 20 14 0 17 Aug 2022
Decentralized Gossip-Based Stochastic Bilevel Optimization over Communication Networks Shuoguang Yang Xuezhou Zhang Mengdi Wang 32 42 0 22 Jun 2022
Analysing the Influence of Attack Configurations on the Reconstruction of Medical Images in Federated Learning M. Dahlgaard Morten Wehlast Jorgensen N. Fuglsang Hiba Nassar FedML AAML 38 2 0 25 Apr 2022
On Acceleration of Gradient-Based Empirical Risk Minimization using Local Polynomial Regression Ekaterina Trimbach Edward Duc Hien Nguyen César A. Uribe 23 1 0 16 Apr 2022
Personalization Improves Privacy-Accuracy Tradeoffs in Federated Learning A. Bietti Chen-Yu Wei Miroslav Dudík John Langford Zhiwei Steven Wu FedML 20 43 0 10 Feb 2022
Addressing modern and practical challenges in machine learning: A survey of online federated and transfer learning Shuang Dai Fanlin Meng FedML OnRL 37 21 0 07 Feb 2022
Federated Optimization of Smooth Loss Functions Ali Jadbabaie A. Makur Devavrat Shah FedML 19 7 0 06 Jan 2022
Stochastic Mirror Descent: Convergence Analysis and Adaptive Variants via the Mirror Stochastic Polyak Stepsize Ryan DÓrazio Nicolas Loizou I. Laradji Ioannis Mitliagkas 34 30 0 28 Oct 2021
A Stochastic Newton Algorithm for Distributed Convex Optimization Brian Bullins Kumar Kshitij Patel Ohad Shamir Nathan Srebro Blake E. Woodworth 26 15 0 07 Oct 2021
Decentralized optimization with non-identical sampling in presence of stragglers Tharindu B. Adikari S. Draper 35 1 0 25 Aug 2021
Opportunistic Emulation of Computationally Expensive Simulations via Deep Learning Conrad Sanderson D. Pagendam B. Power Frederick Bennett Ross Darnell 16 2 0 25 Aug 2021
A Field Guide to Federated Optimization Jianyu Wang Zachary B. Charles Zheng Xu Gauri Joshi H. B. McMahan ... Mi Zhang Tong Zhang Chunxiang Zheng Chen Zhu Wennan Zhu FedML 187 411 0 14 Jul 2021
Estimating Fund-Raising Performance for Start-up Projects from a Market Graph Perspective Likang Wu Zhi Li Hongke Zhao Qi Liu Enhong Chen 36 16 0 27 May 2021
Escaping Saddle Points with Compressed SGD Dmitrii Avdiukhin G. Yaroslavtsev 6 4 0 21 May 2021
Consensus Control for Decentralized Deep Learning Lingjing Kong Tao R. Lin Anastasia Koloskova Martin Jaggi Sebastian U. Stich 19 75 0 09 Feb 2021
Distributed Machine Learning for Wireless Communication Networks: Techniques, Architectures, and Applications Shuyan Hu Xiaojing Chen Wei Ni E. Hossain Xin Wang AI4CE 42 111 0 02 Dec 2020
Federated Composite Optimization Honglin Yuan Manzil Zaheer Sashank J. Reddi FedML 32 58 0 17 Nov 2020
Gradient-Based Empirical Risk Minimization using Local Polynomial Regression Ali Jadbabaie A. Makur Devavrat Shah 28 6 0 04 Nov 2020
Async-RED: A Provably Convergent Asynchronous Block Parallel Stochastic Method using Deep Denoising Priors Yu Sun Jiaming Liu Yiran Sun B. Wohlberg Ulugbek S. Kamilov 34 15 0 03 Oct 2020
Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applications Jihong Park S. Samarakoon Anis Elgabli Joongheon Kim M. Bennis Seong-Lyun Kim Mérouane Debbah 34 161 0 06 Aug 2020
Fast-Convergent Federated Learning Hung T. Nguyen Vikash Sehwag Seyyedali Hosseinalipour Christopher G. Brinton M. Chiang H. Vincent Poor FedML 24 191 0 26 Jul 2020
FedSplit: An algorithmic framework for fast federated optimization Reese Pathak Martin J. Wainwright FedML 39 182 0 11 May 2020
Federated Residual Learning Alekh Agarwal John Langford Chen-Yu Wei FedML 17 40 0 28 Mar 2020
A Unified Theory of Decentralized SGD with Changing Topology and Local Updates Anastasia Koloskova Nicolas Loizou Sadra Boreiri Martin Jaggi Sebastian U. Stich FedML 41 491 0 23 Mar 2020
Communication optimization strategies for distributed deep neural network training: A survey Shuo Ouyang Dezun Dong Yemao Xu Liquan Xiao 24 12 0 06 Mar 2020
Communication-Efficient Decentralized Learning with Sparsification and Adaptive Peer Selection Zhenheng Tang S. Shi X. Chu FedML 18 57 0 22 Feb 2020
Is Local SGD Better than Minibatch SGD? Blake E. Woodworth Kumar Kshitij Patel Sebastian U. Stich Zhen Dai Brian Bullins H. B. McMahan Ohad Shamir Nathan Srebro FedML 34 253 0 18 Feb 2020
Better Theory for SGD in the Nonconvex World Ahmed Khaled Peter Richtárik 13 178 0 09 Feb 2020
Variance Reduced Local SGD with Lower Communication Complexity Xian-Feng Liang Shuheng Shen Jingchang Liu Zhen Pan Enhong Chen Yifei Cheng FedML 24 152 0 30 Dec 2019
Layer-wise Adaptive Gradient Sparsification for Distributed Deep Learning with Convergence Guarantees S. Shi Zhenheng Tang Qiang-qiang Wang Kaiyong Zhao X. Chu 11 22 0 20 Nov 2019
Unifying mirror descent and dual averaging A. Juditsky Joon Kwon Eric Moulines 9 29 0 30 Oct 2019
Minimax Regret of Switching-Constrained Online Convex Optimization: No Phase Transition Lin Chen Qian-long Yu Hannah Lawrence Amin Karbasi 22 20 0 24 Oct 2019
Complexity of Highly Parallel Non-Smooth Convex Optimization Sébastien Bubeck Qijia Jiang Y. Lee Yuanzhi Li Aaron Sidford 20 55 0 25 Jun 2019
The Step Decay Schedule: A Near Optimal, Geometrically Decaying Learning Rate Procedure For Least Squares Rong Ge Sham Kakade Rahul Kidambi Praneeth Netrapalli 21 149 0 29 Apr 2019
Distributed Learning with Sublinear Communication Jayadev Acharya Christopher De Sa Dylan J. Foster Karthik Sridharan FedML 21 40 0 28 Feb 2019
First-order Newton-type Estimator for Distributed Estimation and Inference Xi Chen Weidong Liu Yichen Zhang 27 48 0 28 Nov 2018
Adaptive Communication Strategies to Achieve the Best Error-Runtime Trade-off in Local-Update SGD Jianyu Wang Gauri Joshi FedML 25 231 0 19 Oct 2018