Random Reshuffling: Simple Analysis with Vast Improvements

10 June 2020

Papers citing "Random Reshuffling: Simple Analysis with Vast Improvements"

30 / 30 papers shown

Title
Better Rates for Random Task Orderings in Continual Linear Models Itay Evron Ran Levinstein Matan Schliserman Uri Sherman Tomer Koren Daniel Soudry Nathan Srebro CLL 35 0 0 06 Apr 2025
Variance Reduction Methods Do Not Need to Compute Full Gradients: Improved Efficiency through Shuffling Daniil Medyakov Gleb Molodtsov S. Chezhegov Alexey Rebrikov Aleksandr Beznosikov 96 0 0 21 Feb 2025
Random Reshuffling for Stochastic Gradient Langevin Dynamics Luke Shaw Peter A. Whalley 82 3 0 28 Jan 2025
How Neural Networks Learn the Support is an Implicit Regularization Effect of SGD Pierfrancesco Beneventano Andrea Pinto Tomaso A. Poggio MLT 29 1 0 17 Jun 2024
Demystifying SGD with Doubly Stochastic Gradients Kyurae Kim Joohwan Ko Yian Ma Jacob R. Gardner 53 0 0 03 Jun 2024
High Probability Guarantees for Random Reshuffling Hengxu Yu Xiao Li 37 2 0 20 Nov 2023
Convergence of Sign-based Random Reshuffling Algorithms for Nonconvex Optimization Zhen Qin Zhishuai Liu Pan Xu 18 1 0 24 Oct 2023
Demystifying the Myths and Legends of Nonconvex Convergence of SGD Aritra Dutta El Houcine Bergou Soumia Boucherouite Nicklas Werge M. Kandemir Xin Li 26 0 0 19 Oct 2023
On The Impact of Machine Learning Randomness on Group Fairness Prakhar Ganesh Hong Chang Martin Strobel Reza Shokri FaML 30 30 0 09 Jul 2023
Distributed Random Reshuffling Methods with Improved Convergence Kun-Yen Huang Linli Zhou Shi Pu 24 4 0 21 Jun 2023
First Order Methods with Markovian Noise: from Acceleration to Variational Inequalities Aleksandr Beznosikov S. Samsonov Marina Sheshukova Alexander Gasnikov A. Naumov Eric Moulines 32 14 0 25 May 2023
Tighter Lower Bounds for Shuffling SGD: Random Permutations and Beyond Jaeyoung Cha Jaewook Lee Chulhee Yun 21 23 0 13 Mar 2023
Distributed Stochastic Optimization under a General Variance Condition Kun-Yen Huang Xiao Li Shin-Yi Pu FedML 40 6 0 30 Jan 2023
Convergence of First-Order Algorithms for Meta-Learning with Moreau Envelopes Konstantin Mishchenko Slavomír Hanzely Peter Richtárik FedML 24 5 0 17 Jan 2023
Momentum Aggregation for Private Non-convex ERM Hoang Tran Ashok Cutkosky 15 14 0 12 Oct 2022
Towards Communication-efficient Vertical Federated Learning Training via Cache-enabled Local Updates Fangcheng Fu Xupeng Miao Jiawei Jiang Huanran Xue Bin Cui FedML 22 21 0 29 Jul 2022
SPIRAL: A superlinearly convergent incremental proximal algorithm for nonconvex finite sum minimization Pourya Behmandpoor P. Latafat Andreas Themelis Marc Moonen Panagiotis Patrinos 24 2 0 17 Jul 2022
On the Convergence to a Global Solution of Shuffling-Type Gradient Algorithms Lam M. Nguyen Trang H. Tran 32 2 0 13 Jun 2022
GraB: Finding Provably Better Data Permutations than Random Reshuffling Yucheng Lu Wentao Guo Christopher De Sa FedML 18 16 0 22 May 2022
Federated Random Reshuffling with Compression and Variance Reduction Grigory Malinovsky Peter Richtárik FedML 22 10 0 08 May 2022
Benign Underfitting of Stochastic Gradient Descent Tomer Koren Roi Livni Yishay Mansour Uri Sherman MLT 15 13 0 27 Feb 2022
Nesterov Accelerated Shuffling Gradient Method for Convex Optimization Trang H. Tran K. Scheinberg Lam M. Nguyen 35 11 0 07 Feb 2022
Distributed Random Reshuffling over Networks Kun-Yen Huang Xiao Li Andre Milzarek Shi Pu Junwen Qiu 36 11 0 31 Dec 2021
Distributed stochastic proximal algorithm with random reshuffling for non-smooth finite-sum optimization Xia Jiang Xianlin Zeng Jian Sun Jie Chen Lihua Xie 13 6 0 06 Nov 2021
Random Shuffling Beats SGD Only After Many Epochs on Ill-Conditioned Problems Itay Safran Ohad Shamir 20 19 0 12 Jun 2021
Permutation-Based SGD: Is Random Optimal? Shashank Rajput Kangwook Lee Dimitris Papailiopoulos 23 14 0 19 Feb 2021
Distributed Second Order Methods with Fast Rates and Compressed Communication Rustem Islamov Xun Qian Peter Richtárik 21 51 0 14 Feb 2021
SMG: A Shuffling Gradient-Based Method with Momentum Trang H. Tran Lam M. Nguyen Quoc Tran-Dinh 13 21 0 24 Nov 2020
Incremental Without Replacement Sampling in Nonconvex Optimization Edouard Pauwels 27 5 0 15 Jul 2020
A Unified Convergence Analysis for Shuffling-Type Gradient Methods Lam M. Nguyen Quoc Tran-Dinh Dzung Phan Phuong Ha Nguyen Marten van Dijk 31 78 0 19 Feb 2020