Optimization Methods for Large-Scale Machine Learning

15 June 2016

Papers citing "Optimization Methods for Large-Scale Machine Learning"

50 / 1,406 papers shown

Title
Never Skip a Batch: Continuous Training of Temporal GNNs via Adaptive Pseudo-Supervision Alexander Panyshev Dmitry Vinichenko Oleg Travkin Roman Alferov Alexey Zaytsev 2 0 0 18 May 2025
Dynamic Perturbed Adaptive Method for Infinite Task-Conflicting Time Series Jiang You Xiaozhen Wang Arben Cela AI4TS 16 0 0 17 May 2025
$On the $O(\frac{\sqrt{d}}{K^{1/4}})$ Convergence Rate of AdamW Measured by $\ell_1$ Norm$ On the $O(\frac{\sqrt{d}}{K^{1/4}})$ Convergence Rate of AdamW Measured by $\ell_1$ Norm Huan Li Yiming Dong Zhouchen Lin 6 0 0 17 May 2025
Sharp Gaussian approximations for Decentralized Federated Learning Soham Bonnerjee Sayar Karmakar Wei Biao Wu FedML 26 0 0 12 May 2025
A stochastic gradient method for trilevel optimization Tommaso Giovannelli G. Kent Luis Nunes Vicente 34 0 0 11 May 2025
Entropy-Guided Sampling of Flat Modes in Discrete Spaces Pinaki Mohanty Riddhiman Bhattacharya Ruqi Zhang 161 0 0 05 May 2025
Online Functional Principal Component Analysis on a Multidimensional Domain Muye Nanshan Nan Zhang Jiguo Cao 21 0 0 04 May 2025
Accelerating Deep Neural Network Training via Distributed Hybrid Order Optimization Shunxian Gu Chaoqun You Bangbang Ren Lailong Luo Junxu Xia Deke Guo 44 0 0 02 May 2025
Ultra-fast feature learning for the training of two-layer neural networks in the two-timescale regime Raphael Barboni Gabriel Peyré François-Xavier Vialard MLT 39 0 0 25 Apr 2025
TACO: Tackling Over-correction in Federated Learning with Tailored Adaptive Correction Weijie Liu Ziwei Zhan Carlee Joe-Wong Edith Ngai Jingpu Duan Deke Guo Xu Chen Xinsong Zhang FedML 53 0 0 24 Apr 2025
OptimAI: Optimization from Natural Language Using LLM-Powered AI Agents Raghav Thind Youran Sun Ling Liang Haizhao Yang LLMAG 36 0 0 23 Apr 2025
AlphaGrad: Non-Linear Gradient Normalization Optimizer Soham Sane ODL 56 0 0 22 Apr 2025
MetaMolGen: A Neural Graph Motif Generation Model for De Novo Molecular Design Zimo Yan Jie Zhang Zheng Xie Chang-rui Liu Yong-Jin Liu Yiping Song 42 0 0 22 Apr 2025
Mixed-Precision Conjugate Gradient Solvers with RL-Driven Precision Tuning Xinye Chen 22 0 0 19 Apr 2025
Stochastic Gradient Descent in Non-Convex Problems: Asymptotic Convergence with Relaxed Step-Size via Stopping Time Methods Ruinan Jin Difei Cheng Hong Qiao Xin Shi Shaodong Liu Bo Zhang 31 0 0 17 Apr 2025
Second-order Optimization of Gaussian Splats with Importance Sampling Hamza Pehlivan Andrea Boscolo Camiletto Lin Geng Foo Marc Habermann Christian Theobalt 3DGS 25 0 0 17 Apr 2025
Towards Weaker Variance Assumptions for Stochastic Optimization Ahmet Alacaoglu Yura Malitsky Stephen J. Wright 33 0 0 14 Apr 2025
A Tale of Two Learning Algorithms: Multiple Stream Random Walk and Asynchronous Gossip Peyman Gholami H. Seferoglu 34 0 0 14 Apr 2025
A Piecewise Lyapunov Analysis of Sub-quadratic SGD: Applications to Robust and Quantile Regression Yixuan Zhang Dongyan Yudong Chen Qiaomin Xie 29 0 0 11 Apr 2025
Min-Max Optimisation for Nonconvex-Nonconcave Functions Using a Random Zeroth-Order Extragradient Algorithm Amir Ali Farzin Yuen-Man Pun Philipp Braun Antoine Lesage-Landry Youssef Diouane Iman Shames 50 1 0 10 Apr 2025
ZIP: An Efficient Zeroth-order Prompt Tuning for Black-box Vision-Language Models Seonghwan Park Jaehyeon Jeong Yongjun Kim Jaeho Lee Namhoon Lee VLM 50 0 0 09 Apr 2025
Decentralized Federated Domain Generalization with Style Sharing: A Formal Modeling and Convergence Analysis Shahryar Zehtabi Dong-Jun Han Seyyedali Hosseinalipour Christopher G. Brinton FedML AI4CE 50 0 0 08 Apr 2025
Universal Collection of Euclidean Invariants between Pairs of Position-Orientations Gijs Bellaard B. Smets R. Duits 64 0 0 04 Apr 2025
Approximate Agreement Algorithms for Byzantine Collaborative Learning Tijana Milentijević Mélanie Cambus Darya Melnyk Stefan Schmid FedML 52 0 0 02 Apr 2025
Unlocking the Value of Decentralized Data: A Federated Dual Learning Approach for Model Aggregation Junyi Zhu Ruicong Yao Taha Ceritli Savas Ozkan Matthew B. Blaschko Eunchung Noh Jeongwon Min Cho Jung Min Mete Ozay FedML 103 0 0 26 Mar 2025
A Flexible Fairness Framework with Surrogate Loss Reweighting for Addressing Sociodemographic Disparities Wen Xu Elham Dolatabadi FaML 89 0 0 21 Mar 2025
From Denoising Score Matching to Langevin Sampling: A Fine-Grained Error Analysis in the Gaussian Setting Samuel Hurault M. Terris Thomas Moreau Gabriel Peyré DiffM 43 1 0 14 Mar 2025
Decision-Dependent Stochastic Optimization: The Role of Distribution Dynamics Zhiyu He S. Bolognani Florian Dorfler Michael Muehlebach 69 0 0 10 Mar 2025
Learning Energy-Based Models by Self-normalising the Likelihood Hugo Senetaire Paul Jeha Pierre-Alexandre Mattei J. Frellsen 44 0 0 10 Mar 2025
FUSE: First-Order and Second-Order Unified SynthEsis in Stochastic Optimization Zhanhong Jiang Md Zahid Hasan Aditya Balu Joshua R. Waite Genyi Huang S. Sarkar 52 0 0 06 Mar 2025
SASSHA: Sharpness-aware Adaptive Second-order Optimization with Stable Hessian Approximation Dahun Shin Dongyeop Lee Jinseok Chung Namhoon Lee ODL AAML 192 0 0 25 Feb 2025
Convergence of Shallow ReLU Networks on Weakly Interacting Data Léo Dana Francis R. Bach Loucas Pillaud-Vivien MLT 62 1 0 24 Feb 2025
Theory-guided Pseudo-spectral Full Waveform Inversion via Deep Neural Networks Christopher Zerafa Pauline Galea Cristiana Sebu 62 0 0 24 Feb 2025
Straight to Zero: Why Linearly Decaying the Learning Rate to Zero Works Best for LLMs Shane Bergsma Nolan Dey Gurpreet Gosal Gavia Gray Daria Soboleva Joel Hestness 58 5 0 21 Feb 2025
Verification and Validation for Trustworthy Scientific Machine Learning John D. Jakeman Lorena A. Barba J. Martins Thomas O'Leary-Roseberry AI4CE 58 0 0 21 Feb 2025
Analog In-memory Training on General Non-ideal Resistive Elements: The Impact of Response Functions Zhaoxian Wu Quan Xian Tayfun Gokmen Omobayode Fagbohungbe Tianyi Chen 91 0 0 17 Feb 2025
Preconditioned Inexact Stochastic ADMM for Deep Model Shenglong Zhou Ouya Wang Ziyan Luo Yongxu Zhu Geoffrey Ye Li 46 0 0 15 Feb 2025
Comparison of CNN-based deep learning architectures for unsteady CFD acceleration on small datasets Sangam Khanal Shilaj Baral Joongoo Jeon AI4CE 66 1 0 06 Feb 2025
Randomized Block-Coordinate Optimistic Gradient Algorithms for Root-Finding Problems Quoc Tran-Dinh Yang Luo 94 6 0 28 Jan 2025
PBM-VFL: Vertical Federated Learning with Feature and Sample Privacy Linh Tran Timothy Castiglia Stacy Patterson Ana Milanova FedML 45 0 0 23 Jan 2025
Learning Versatile Optimizers on a Compute Diet A. Moudgil Boris Knyazev Guillaume Lajoie Eugene Belilovsky 162 0 0 22 Jan 2025
Preconditioned Sharpness-Aware Minimization: Unifying Analysis and a Novel Learning Algorithm Yilang Zhang Bingcong Li G. Giannakis AAML 39 0 0 11 Jan 2025
Revisiting LocalSGD and SCAFFOLD: Improved Rates and Missing Analysis Ruichen Luo Sebastian U Stich Samuel Horváth Martin Takáč 40 0 0 08 Jan 2025
Adaptive Batch Size Schedules for Distributed Training of Language Models with Data and Model Parallelism Tim Tsz-Kit Lau Weijian Li Chenwei Xu Han Liu Mladen Kolar 159 0 0 30 Dec 2024
MARINA-P: Superior Performance in Non-smooth Federated Optimization with Adaptive Stepsizes Igor Sokolov Peter Richtárik 77 1 0 22 Dec 2024
Sharpness-Aware Minimization with Adaptive Regularization for Training Deep Neural Networks Jinping Zou Xiaoge Deng Tao Sun 79 0 0 22 Dec 2024
Causal Invariance Learning via Efficient Optimization of a Nonconvex Objective Zhenyu Wang Yifan Hu Peter Buhlmann Zijian Guo 108 1 0 16 Dec 2024
Towards Understanding the Role of Sharpness-Aware Minimization Algorithms for Out-of-Distribution Generalization Samuel Schapiro Han Zhao 76 1 0 06 Dec 2024
Conformal Symplectic Optimization for Stable Reinforcement Learning Yao Lyu Xiangteng Zhang Shengbo Eben Li Jingliang Duan Letian Tao Qing Xu Lei He Keqiang Li 68 0 0 03 Dec 2024
Curvature in the Looking-Glass: Optimal Methods to Exploit Curvature of Expectation in the Loss Landscape Jed A. Duersch Tommie A. Catanach Alexander Safonov Jeremy Wendt 84 0 0 25 Nov 2024