Optimal Distributed Online Prediction using Mini-Batches

7 December 2010

Papers citing "Optimal Distributed Online Prediction using Mini-Batches"

50 / 85 papers shown

Title
Weight for Robustness: A Comprehensive Approach towards Optimal Fault-Tolerant Asynchronous ML T. Dahan Kfir Y. Levy 65 0 0 17 Jan 2025
Revisiting LocalSGD and SCAFFOLD: Improved Rates and Missing Analysis Ruichen Luo Sebastian U Stich Samuel Horváth Martin Takáč 38 0 0 08 Jan 2025
Faster Stochastic Optimization with Arbitrary Delays via Asynchronous Mini-Batching Amit Attia Ofir Gaash Tomer Koren 38 0 0 14 Aug 2024
The Limits and Potentials of Local SGD for Distributed Heterogeneous Learning with Intermittent Communication Kumar Kshitij Patel Margalit Glasgow Ali Zindari Lingxiao Wang Sebastian U. Stich Ziheng Cheng Nirmit Joshi Nathan Srebro 44 6 0 19 May 2024
Stochastic Online Optimization for Cyber-Physical and Robotic Systems Hao Ma M. Zeilinger Michael Muehlebach 44 0 0 08 Apr 2024
Federated Multi-Level Optimization over Decentralized Networks Shuoguang Yang Xuezhou Zhang Mengdi Wang AI4CE 28 0 0 10 Oct 2023
DropCompute: simple and more robust distributed synchronous training via compute variance reduction Niv Giladi Shahar Gottlieb Moran Shkolnik A. Karnieli Ron Banner Elad Hoffer Kfir Y. Levy Daniel Soudry 25 2 0 18 Jun 2023
Repeated Random Sampling for Minimizing the Time-to-Accuracy of Learning Patrik Okanovic R. Waleffe Vasilis Mageirakos Konstantinos E. Nikolakakis Amin Karbasi Dionysis Kalogerias Nezihe Merve Gürel Theodoros Rekatsinas DD 39 12 0 28 May 2023
SLowcal-SGD: Slow Query Points Improve Local-SGD for Stochastic Convex Optimization Kfir Y. Levy Kfir Y. Levy FedML 43 2 0 09 Apr 2023
Two Losses Are Better Than One: Faster Optimization Using a Cheaper Proxy Blake E. Woodworth Konstantin Mishchenko Francis R. Bach 42 6 0 07 Feb 2023
ReSQueing Parallel and Private Stochastic Convex Optimization Y. Carmon A. Jambulapati Yujia Jin Y. Lee Daogao Liu Aaron Sidford Kevin Tian FedML 10 12 0 01 Jan 2023
RCD-SGD: Resource-Constrained Distributed SGD in Heterogeneous Environment via Submodular Partitioning Haoze He Parijat Dube 13 1 0 02 Nov 2022
NET-FLEET: Achieving Linear Convergence Speedup for Fully Decentralized Federated Learning with Heterogeneous Data Xin Zhang Minghong Fang Zhuqing Liu Haibo Yang Jia-Wei Liu Zhengyuan Zhu FedML 13 14 0 17 Aug 2022
Decentralized Gossip-Based Stochastic Bilevel Optimization over Communication Networks Shuoguang Yang Xuezhou Zhang Mengdi Wang 32 42 0 22 Jun 2022
Analysing the Influence of Attack Configurations on the Reconstruction of Medical Images in Federated Learning M. Dahlgaard Morten Wehlast Jorgensen N. Fuglsang Hiba Nassar FedML AAML 28 2 0 25 Apr 2022
On Acceleration of Gradient-Based Empirical Risk Minimization using Local Polynomial Regression Ekaterina Trimbach Edward Duc Hien Nguyen César A. Uribe 18 1 0 16 Apr 2022
Personalization Improves Privacy-Accuracy Tradeoffs in Federated Learning A. Bietti Chen-Yu Wei Miroslav Dudík John Langford Zhiwei Steven Wu FedML 20 43 0 10 Feb 2022
Addressing modern and practical challenges in machine learning: A survey of online federated and transfer learning Shuang Dai Fanlin Meng FedML OnRL 34 21 0 07 Feb 2022
Federated Optimization of Smooth Loss Functions Ali Jadbabaie A. Makur Devavrat Shah FedML 17 7 0 06 Jan 2022
Stochastic Mirror Descent: Convergence Analysis and Adaptive Variants via the Mirror Stochastic Polyak Stepsize Ryan DÓrazio Nicolas Loizou I. Laradji Ioannis Mitliagkas 29 30 0 28 Oct 2021
A Stochastic Newton Algorithm for Distributed Convex Optimization Brian Bullins Kumar Kshitij Patel Ohad Shamir Nathan Srebro Blake E. Woodworth 26 15 0 07 Oct 2021
Decentralized optimization with non-identical sampling in presence of stragglers Tharindu B. Adikari S. Draper 35 1 0 25 Aug 2021
Opportunistic Emulation of Computationally Expensive Simulations via Deep Learning Conrad Sanderson D. Pagendam B. Power Frederick Bennett Ross Darnell 16 2 0 25 Aug 2021
A Field Guide to Federated Optimization Jianyu Wang Zachary B. Charles Zheng Xu Gauri Joshi H. B. McMahan ... Mi Zhang Tong Zhang Chunxiang Zheng Chen Zhu Wennan Zhu FedML 178 411 0 14 Jul 2021
Estimating Fund-Raising Performance for Start-up Projects from a Market Graph Perspective Likang Wu Zhi Li Hongke Zhao Qi Liu Enhong Chen 28 16 0 27 May 2021
Consensus Control for Decentralized Deep Learning Lingjing Kong Tao R. Lin Anastasia Koloskova Martin Jaggi Sebastian U. Stich 19 75 0 09 Feb 2021
Distributed Machine Learning for Wireless Communication Networks: Techniques, Architectures, and Applications Shuyan Hu Xiaojing Chen Wei Ni E. Hossain Xin Wang AI4CE 39 111 0 02 Dec 2020
Federated Composite Optimization Honglin Yuan Manzil Zaheer Sashank J. Reddi FedML 24 57 0 17 Nov 2020
Gradient-Based Empirical Risk Minimization using Local Polynomial Regression Ali Jadbabaie A. Makur Devavrat Shah 20 6 0 04 Nov 2020
Async-RED: A Provably Convergent Asynchronous Block Parallel Stochastic Method using Deep Denoising Priors Yu Sun Jiaming Liu Yiran Sun B. Wohlberg Ulugbek S. Kamilov 22 15 0 03 Oct 2020
Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applications Jihong Park S. Samarakoon Anis Elgabli Joongheon Kim M. Bennis Seong-Lyun Kim Mérouane Debbah 25 161 0 06 Aug 2020
Fast-Convergent Federated Learning Hung T. Nguyen Vikash Sehwag Seyyedali Hosseinalipour Christopher G. Brinton M. Chiang H. Vincent Poor FedML 24 191 0 26 Jul 2020
FedSplit: An algorithmic framework for fast federated optimization Reese Pathak Martin J. Wainwright FedML 35 182 0 11 May 2020
Federated Residual Learning Alekh Agarwal John Langford Chen-Yu Wei FedML 12 40 0 28 Mar 2020
A Unified Theory of Decentralized SGD with Changing Topology and Local Updates Anastasia Koloskova Nicolas Loizou Sadra Boreiri Martin Jaggi Sebastian U. Stich FedML 39 491 0 23 Mar 2020
Communication optimization strategies for distributed deep neural network training: A survey Shuo Ouyang Dezun Dong Yemao Xu Liquan Xiao 17 12 0 06 Mar 2020
Communication-Efficient Decentralized Learning with Sparsification and Adaptive Peer Selection Zhenheng Tang S. Shi X. Chu FedML 13 57 0 22 Feb 2020
Is Local SGD Better than Minibatch SGD? Blake E. Woodworth Kumar Kshitij Patel Sebastian U. Stich Zhen Dai Brian Bullins H. B. McMahan Ohad Shamir Nathan Srebro FedML 34 253 0 18 Feb 2020
Better Theory for SGD in the Nonconvex World Ahmed Khaled Peter Richtárik 13 178 0 09 Feb 2020
Variance Reduced Local SGD with Lower Communication Complexity Xian-Feng Liang Shuheng Shen Jingchang Liu Zhen Pan Enhong Chen Yifei Cheng FedML 24 152 0 30 Dec 2019
Minimax Regret of Switching-Constrained Online Convex Optimization: No Phase Transition Lin Chen Qian-long Yu Hannah Lawrence Amin Karbasi 16 20 0 24 Oct 2019
Complexity of Highly Parallel Non-Smooth Convex Optimization Sébastien Bubeck Qijia Jiang Y. Lee Yuanzhi Li Aaron Sidford 15 55 0 25 Jun 2019
The Step Decay Schedule: A Near Optimal, Geometrically Decaying Learning Rate Procedure For Least Squares Rong Ge Sham Kakade Rahul Kidambi Praneeth Netrapalli 17 149 0 29 Apr 2019
First-order Newton-type Estimator for Distributed Estimation and Inference Xi Chen Weidong Liu Yichen Zhang 22 48 0 28 Nov 2018
Effective Parallelisation for Machine Learning Michael Kamp Mario Boley Olana Missura Thomas Gärtner 11 12 0 08 Oct 2018
Anytime Stochastic Gradient Descent: A Time to Hear from all the Workers Nuwan S. Ferdinand S. Draper 8 19 0 06 Oct 2018
Graph-Dependent Implicit Regularisation for Distributed Stochastic Subgradient Descent Dominic Richards Patrick Rebeschini 14 18 0 18 Sep 2018
Don't Use Large Mini-Batches, Use Local SGD Tao R. Lin Sebastian U. Stich Kumar Kshitij Patel Martin Jaggi 34 429 0 22 Aug 2018
Parallelization does not Accelerate Convex Optimization: Adaptivity Lower Bounds for Non-smooth Convex Minimization Eric Balkanski Yaron Singer 14 31 0 12 Aug 2018
Efficient Decentralized Deep Learning by Dynamic Model Averaging Michael Kamp Linara Adilova Joachim Sicking Fabian Hüger Peter Schlicht Tim Wirtz Stefan Wrobel 19 128 0 09 Jul 2018