On the Ineffectiveness of Variance Reduced Optimization for Deep Learning

11 December 2018

Papers citing "On the Ineffectiveness of Variance Reduced Optimization for Deep Learning"

22 / 22 papers shown

Title
Trial and Trust: Addressing Byzantine Attacks with Comprehensive Defense Strategy Gleb Molodtsov Daniil Medyakov Sergey Skorik Nikolas Khachaturov Shahane Tigranyan Vladimir Aletov A. Avetisyan Martin Takáč Aleksandr Beznosikov AAML 35 0 0 12 May 2025
Convergence Conditions for Stochastic Line Search Based Optimization of Over-parametrized Models Matteo Lapucci Davide Pucci 35 1 0 06 Aug 2024
Deep Companion Learning: Enhancing Generalization Through Historical Consistency Ruizhao Zhu Venkatesh Saligrama FedML 34 0 0 26 Jul 2024
Faster Convergence of Stochastic Accelerated Gradient Descent under Interpolation Aaron Mishkin Mert Pilanci Mark Schmidt 64 1 0 03 Apr 2024
A Coefficient Makes SVRG Effective Yida Yin Zhiqiu Xu Zhiyuan Li Trevor Darrell Zhuang Liu 33 1 0 09 Nov 2023
Nonconvex Stochastic Bregman Proximal Gradient Method with Application to Deep Learning Kuan-Fu Ding Jingyang Li Kim-Chuan Toh 30 8 0 26 Jun 2023
Statistically Optimal Force Aggregation for Coarse-Graining Molecular Dynamics Andreas Krämer Aleksander E. P. Durumeric N. Charron Yaoyi Chen C. Clementi Frank Noé AI4CE 27 20 0 14 Feb 2023
On the effectiveness of partial variance reduction in federated learning with heterogeneous data Bo-wen Li Mikkel N. Schmidt T. S. Alstrøm Sebastian U. Stich FedML 37 9 0 05 Dec 2022
Closing the Generalization Gap of Cross-silo Federated Medical Image Segmentation An Xu Wenqi Li Pengfei Guo Dong Yang H. Roth Ali Hatamizadeh Can Zhao Daguang Xu Heng-Chiao Huang Ziyue Xu FedML 36 51 0 18 Mar 2022
Tackling benign nonconvexity with smoothing and stochastic gradients Harsh Vardhan Sebastian U. Stich 26 8 0 18 Feb 2022
Training Structured Neural Networks Through Manifold Identification and Variance Reduction Zih-Syuan Huang Ching-pei Lee AAML 46 9 0 05 Dec 2021
Secure Distributed Training at Scale Eduard A. Gorbunov Alexander Borzunov Michael Diskin Max Ryabinin FedML 21 15 0 21 Jun 2021
Variance Reduced Training with Stratified Sampling for Forecasting Models Yucheng Lu Youngsuk Park Lifan Chen Bernie Wang Christopher De Sa Dean Phillips Foster AI4TS 38 17 0 02 Mar 2021
SVRG Meets AdaGrad: Painless Variance Reduction Benjamin Dubois-Taine Sharan Vaswani Reza Babanezhad Mark W. Schmidt Simon Lacoste-Julien 18 18 0 18 Feb 2021
Iterative Averaging in the Quest for Best Test Error Diego Granziol Xingchen Wan Samuel Albanie Stephen J. Roberts 10 3 0 02 Mar 2020
Adaptive Federated Optimization Sashank J. Reddi Zachary B. Charles Manzil Zaheer Zachary Garrett Keith Rush Jakub Konecný Sanjiv Kumar H. B. McMahan FedML 15 1,391 0 29 Feb 2020
Optimization for deep learning: theory and algorithms Ruoyu Sun ODL 14 168 0 19 Dec 2019
Lookahead Optimizer: k steps forward, 1 step back Michael Ruogu Zhang James Lucas Geoffrey E. Hinton Jimmy Ba ODL 33 719 0 19 Jul 2019
Why gradient clipping accelerates training: A theoretical justification for adaptivity Junzhe Zhang Tianxing He S. Sra Ali Jadbabaie 30 442 0 28 May 2019
Reducing Noise in GAN Training with Variance Reduced Extragradient Tatjana Chavdarova Gauthier Gidel F. Fleuret Simon Lacoste-Julien 25 134 0 18 Apr 2019
Don't Use Large Mini-Batches, Use Local SGD Tao R. Lin Sebastian U. Stich Kumar Kshitij Patel Martin Jaggi 54 429 0 22 Aug 2018
Incremental Majorization-Minimization Optimization with Application to Large-Scale Machine Learning Julien Mairal 79 317 0 18 Feb 2014