Variance-Reduced Methods for Machine Learning

2 October 2020

Papers citing "Variance-Reduced Methods for Machine Learning"

50 / 59 papers shown

Title
AutoSGD: Automatic Learning Rate Selection for Stochastic Gradient Descent Nikola Surjanovic Alexandre Bouchard-Côté Trevor Campbell 20 0 0 27 May 2025
LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Models Fengqi Zhu Rongzhen Wang Shen Nie Xiaolu Zhang Chunwei Wu ... Jun Zhou Jianfei Chen Yankai Lin Ji-Rong Wen Chongxuan Li 190 2 0 25 May 2025
Variance Reduction Methods Do Not Need to Compute Full Gradients: Improved Efficiency through Shuffling Daniil Medyakov Gleb Molodtsov S. Chezhegov Alexey Rebrikov Aleksandr Beznosikov 147 0 0 21 Feb 2025
Analysis of regularized federated learning Langming Liu Dingxuan Zhou FedML 31 0 0 03 Nov 2024
Efficient Optimization Algorithms for Linear Adversarial Training Antônio H. Ribeiro Thomas B. Schon Dave Zahariah Francis Bach AAML 107 2 0 16 Oct 2024
Gap-Dependent Bounds for Q-Learning using Reference-Advantage Decomposition Zhong Zheng Haochen Zhang Lingzhou Xue OffRL 136 4 0 10 Oct 2024
Dataset Distillation via Knowledge Distillation: Towards Efficient Self-Supervised Pre-Training of Deep Networks S. Joshi Jiayi Ni Baharan Mirzasoleiman DD 180 2 0 03 Oct 2024
On the SAGA algorithm with decreasing step Luis Fredes Bernard Bercu Eméric Gbaguidi 51 1 0 02 Oct 2024
On the Convergence of FedProx with Extrapolation and Inexact Prox Hanmin Li Peter Richtárik FedML 88 1 0 02 Oct 2024
An Effective Dynamic Gradient Calibration Method for Continual Learning Weichen Lin Jiaxiang Chen Ru Huang Huihua Ding CLL 94 0 0 30 Jul 2024
SOREL: A Stochastic Algorithm for Spectral Risks Minimization Yuze Ge Rujun Jiang 65 0 0 19 Jul 2024
Unbiased least squares regression via averaged stochastic gradient descent Nabil Kahalé 78 0 0 26 Jun 2024
Communication-Efficient Adaptive Batch Size Strategies for Distributed Local Gradient Methods Tim Tsz-Kit Lau Weijian Li Chenwei Xu Han Liu Mladen Kolar 84 1 0 20 Jun 2024
Demystifying SGD with Doubly Stochastic Gradients Kyurae Kim Joohwan Ko Yian Ma Jacob R. Gardner 140 2 0 03 Jun 2024
Double Variance Reduction: A Smoothing Trick for Composite Optimization Problems without First-Order Gradient Hao Di Haishan Ye Yueling Zhang Xiangyu Chang Guang Dai Ivor W. Tsang 123 1 0 28 May 2024
A Unified Theory of Stochastic Proximal Point Methods without Smoothness Peter Richtárik Abdurakhmon Sadiev Yury Demidovich 80 4 0 24 May 2024
Streamlining in the Riemannian Realm: Efficient Riemannian Optimization with Loopless Variance Reduction Yury Demidovich Grigory Malinovsky Peter Richtárik 83 3 0 11 Mar 2024
LoCoDL: Communication-Efficient Distributed Learning with Local Training and Compression Laurent Condat Artavazd Maranjyan Peter Richtárik 123 5 0 07 Mar 2024
AdAdaGrad: Adaptive Batch Size Schemes for Adaptive Gradient Methods Tim Tsz-Kit Lau Han Liu Mladen Kolar ODL 70 6 0 17 Feb 2024
Critical Influence of Overparameterization on Sharpness-aware Minimization Sungbin Shin Dongyeop Lee Maksym Andriushchenko Namhoon Lee AAML 158 2 0 29 Nov 2023
A Large Deviations Perspective on Policy Gradient Algorithms Wouter Jongeneel Daniel Kuhn Mengmeng Li 55 1 0 13 Nov 2023
A Coefficient Makes SVRG Effective Yida Yin Zhiqiu Xu Zhiyuan Li Trevor Darrell Zhuang Liu 87 1 0 09 Nov 2023
Balancing Act: Constraining Disparate Impact in Sparse Models Meraj Hashemizadeh Juan Ramirez Rohan Sukumaran G. Farnadi Simon Lacoste-Julien Jose Gallego-Posada 60 6 0 31 Oct 2023
Coreset Markov Chain Monte Carlo Naitong Chen Trevor Campbell 47 4 0 25 Oct 2023
Variance Reduced Halpern Iteration for Finite-Sum Monotone Inclusions Xu Cai Ahmet Alacaoglu Jelena Diakonikolas 99 8 0 04 Oct 2023
Joint Sampling and Optimisation for Inverse Rendering Martin Balint K. Myszkowski Hans-Peter Seidel Gurprit Singh 19 1 0 27 Sep 2023
Linear Convergence of Black-Box Variational Inference: Should We Stick the Landing? Kyurae Kim Yian Ma Jacob R. Gardner 88 7 0 27 Jul 2023
Towards a Better Theoretical Understanding of Independent Subnetwork Training Egor Shulgin Peter Richtárik AI4CE 106 6 0 28 Jun 2023
Adaptive Federated Learning with Auto-Tuned Clients Junhyung Lyle Kim Taha Toghani César A. Uribe Anastasios Kyrillidis FedML 124 6 0 19 Jun 2023
$$\textbf{A}^2\textbf{CiD}^2$: Accelerating Asynchronous Communication in Decentralized Deep Learning$ $\textbf{A}^2\textbf{CiD}^2$ : Accelerating Asynchronous Communication in Decentralized Deep Learning Adel Nabli Eugene Belilovsky Edouard Oyallon 74 7 0 14 Jun 2023
$Sketch-and-Project Meets Newton Method: Global $\mathcal O(k^{-2})$ Convergence with Low-Rank Updates$ Sketch-and-Project Meets Newton Method: Global $\mathcal O(k^{-2})$ Convergence with Low-Rank Updates Slavomír Hanzely 51 7 0 22 May 2023
Unified analysis of SGD-type methods Eduard A. Gorbunov 69 2 0 29 Mar 2023
Byzantine-Robust Loopless Stochastic Variance-Reduced Gradient Nikita Fedin Eduard A. Gorbunov 44 2 0 08 Mar 2023
Stochastic Approximation Beyond Gradient for Signal Processing and Machine Learning Aymeric Dieuleveut G. Fort Eric Moulines Hoi-To Wai 74 12 0 22 Feb 2023
TAMUNA: Doubly Accelerated Distributed Optimization with Local Training, Compression, and Partial Participation Laurent Condat Ivan Agarský Grigory Malinovsky Peter Richtárik FedML 108 4 0 20 Feb 2023
A One-Sample Decentralized Proximal Algorithm for Non-Convex Stochastic Composite Optimization Tesi Xiao Xuxing Chen Krishnakumar Balasubramanian Saeed Ghadimi 94 10 0 20 Feb 2023
Balance is Essence: Accelerating Sparse Training via Adaptive Gradient Correction Bowen Lei Dongkuan Xu Ruqi Zhang Shuren He Bani Mallick 117 6 0 09 Jan 2023
Xtreme Margin: A Tunable Loss Function for Binary Classification Problems Rayan Wali MQ 99 4 0 31 Oct 2022
Provably Doubly Accelerated Federated Learning: The First Theoretically Successful Combination of Local Training and Communication Compression Laurent Condat Ivan Agarský Peter Richtárik FedML 113 17 0 24 Oct 2022
Joint control variate for faster black-box variational inference Xi Wang Tomas Geffner Justin Domke BDL DRL 39 0 0 13 Oct 2022
RECAPP: Crafting a More Efficient Catalyst for Convex Optimization Y. Carmon A. Jambulapati Yujia Jin Aaron Sidford 125 12 0 17 Jun 2022
Variance Reduction is an Antidote to Byzantines: Better Rates, Weaker Assumptions and Communication Compression as a Cherry on the Top Eduard A. Gorbunov Samuel Horváth Peter Richtárik Gauthier Gidel AAML 38 0 0 01 Jun 2022
EF-BV: A Unified Theory of Error Feedback and Variance Reduction Mechanisms for Biased and Unbiased Compression in Distributed Optimization Laurent Condat Kai Yi Peter Richtárik 107 21 0 09 May 2022
A Statistical Analysis of Polyak-Ruppert Averaged Q-learning Xiang Li Wenhao Yang Jiadong Liang Zhihua Zhang Michael I. Jordan 122 17 0 29 Dec 2021
LoSAC: An Efficient Local Stochastic Average Control Method for Federated Optimization Huiming Chen Huandong Wang Quanming Yao Yong Li Depeng Jin Qiang Yang FedML 54 5 0 15 Dec 2021
Nearly Optimal Linear Convergence of Stochastic Primal-Dual Methods for Linear Programming Haihao Lu Jinwen Yang 52 6 0 10 Nov 2021
Gradient Descent on Infinitely Wide Neural Networks: Global Convergence and Generalization Francis R. Bach Lénaïc Chizat MLT 67 24 0 15 Oct 2021
Breaking the Sample Complexity Barrier to Regret-Optimal Model-Free Reinforcement Learning Gen Li Laixi Shi Yuxin Chen Yuejie Chi OffRL 94 54 0 09 Oct 2021
MURANA: A Generic Framework for Stochastic Variance-Reduced Optimization Laurent Condat Peter Richtárik 73 19 0 06 Jun 2021
Random Reshuffling with Variance Reduction: New Analysis and Better Rates Grigory Malinovsky Alibek Sailanbayev Peter Richtárik 49 21 0 19 Apr 2021