Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programming

22 September 2013

Papers citing "Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programming"

36 / 36 papers shown

Title
KerZOO: Kernel Function Informed Zeroth-Order Optimization for Accurate and Accelerated LLM Fine-Tuning Zhendong Mi Qitao Tan Xiaodong Yu Zining Zhu Geng Yuan Shaoyi Huang 120 0 0 24 May 2025
Observability conditions for neural state-space models with eigenvalues and their roots of unity Andrew Gracyk 365 0 0 22 Apr 2025
Stochastic Gradient Descent in Non-Convex Problems: Asymptotic Convergence with Relaxed Step-Size via Stopping Time Methods Ruinan Jin Difei Cheng Hong Qiao Xin Shi Shaodong Liu Bo Zhang 63 0 0 17 Apr 2025
Understanding Gradient Orthogonalization for Deep Learning via Non-Euclidean Trust-Region Optimization Dmitry Kovalev 93 3 0 16 Mar 2025
Gradient-free stochastic optimization for additive models A. Akhavan Alexandre B. Tsybakov 91 0 0 03 Mar 2025
Noise-Aware Algorithm for Heterogeneous Differentially Private Federated Learning Saber Malekmohammadi Yaoliang Yu Yang Cao FedML 106 6 0 17 Feb 2025
Small steps no more: Global convergence of stochastic gradient bandits for arbitrary learning rates Jincheng Mei Bo Dai Alekh Agarwal Sharan Vaswani Anant Raj Csaba Szepesvári Dale Schuurmans 96 0 0 11 Feb 2025
Intelligent Sensing-to-Action for Robust Autonomy at the Edge: Opportunities and Challenges A. R. Trivedi Sina Tayebati Hemant Kumawat Nastaran Darabi Divake Kumar ... Dinithi Jayasuriya Nethmi Jayasinghe Priyadarshini Panda Saibal Mukhopadhyay Kaushik Roy 122 0 0 04 Feb 2025
Ringmaster ASGD: The First Asynchronous SGD with Optimal Time Complexity Artavazd Maranjyan Alexander Tyurin Peter Richtárik 71 3 0 27 Jan 2025
Nonlinear Stochastic Gradient Descent and Heavy-tailed Noise: A Unified Framework and High-probability Guarantees Aleksandar Armacki Shuhua Yu Pranay Sharma Gauri Joshi Dragana Bajović D. Jakovetić S. Kar 81 2 0 17 Oct 2024
Adaptive Batch Size for Privately Finding Second-Order Stationary Points Daogao Liu Kunal Talwar 335 0 0 10 Oct 2024
Extended convexity and smoothness and their applications in deep learning Binchuan Qi Wei Gong Li Li 78 0 0 08 Oct 2024
DiSK: Differentially Private Optimizer with Simplified Kalman Filter for Noise Reduction Xinwei Zhang Zhiqi Bu Borja Balle Mingyi Hong Meisam Razaviyayn Vahab Mirrokni 99 2 0 04 Oct 2024
An Accelerated Algorithm for Stochastic Bilevel Optimization under Unbounded Smoothness Xiaochuan Gong Jie Hao Mingrui Liu 91 2 0 28 Sep 2024
Faster Acceleration for Steepest Descent Site Bai Brian Bullins ODL 65 0 0 28 Sep 2024
LiD-FL: Towards List-Decodable Federated Learning Hong Liu Liren Shan Han Bao Ronghui You Yuhao Yi Jiancheng Lv FedML 114 0 0 09 Aug 2024
Gradient Compressed Sensing: A Query-Efficient Gradient Estimator for High-Dimensional Zeroth-Order Optimization Ruizhong Qiu Hanghang Tong 69 5 0 27 May 2024
Does SGD really happen in tiny subspaces? Minhak Song Kwangjun Ahn Chulhee Yun 90 6 1 25 May 2024
Achieving Dimension-Free Communication in Federated Learning via Zeroth-Order Optimization Zhe Li Bicheng Ying Zidong Liu Chaosheng Dong Haibo Yang FedML 90 3 0 24 May 2024
Almost sure convergence rates of stochastic gradient methods under gradient domination Simon Weissmann Sara Klein Waïss Azizian Leif Döring 58 3 0 22 May 2024
Inexact subgradient methods for semialgebraic functions Jérôme Bolte Tam Le Éric Moulines Edouard Pauwels 65 2 0 30 Apr 2024
DACAD: Domain Adaptation Contrastive Learning for Anomaly Detection in Multivariate Time Series Zahra Zamanzadeh Darban Yiyuan Yang Geoffrey I. Webb Charu C. Aggarwal Qingsong Wen Xiaojun Jia Mahsa Salehi 79 0 0 17 Apr 2024
Convergence Guarantees for RMSProp and Adam in Generalized-smooth Non-convex Optimization with Affine Noise Variance Qi Zhang Yi Zhou Shaofeng Zou 82 5 0 01 Apr 2024
Second-Order Fine-Tuning without Pain for LLMs:A Hessian Informed Zeroth-Order Optimizer Yanjun Zhao Sizhe Dang Haishan Ye Guang Dai Yi Qian Ivor W.Tsang 96 9 0 23 Feb 2024
A PAC-Bayesian Link Between Generalisation and Flat Minima Maxime Haddouche Paul Viallard Umut Simsekli Benjamin Guedj 73 3 0 13 Feb 2024
On Convergence of Adam for Stochastic Optimization under Relaxed Assumptions Yusu Hong Junhong Lin 76 13 0 06 Feb 2024
Non-asymptotic Analysis of Biased Adaptive Stochastic Approximation Sobihan Surendran Antoine Godichon-Baggioni Adeline Fermanian Sylvain Le Corff 77 1 0 05 Feb 2024
Private Fine-tuning of Large Language Models with Zeroth-order Optimization Xinyu Tang Ashwinee Panda Milad Nasr Saeed Mahloujifar Prateek Mittal 103 20 0 09 Jan 2024
Deterministic Nonsmooth Nonconvex Optimization Michael I. Jordan Guy Kornowski Tianyi Lin Ohad Shamir Manolis Zampetakis 80 26 0 16 Feb 2023
Adaptive Client Sampling in Federated Learning via Online Learning with Bandit Feedback Boxin Zhao Lingxiao Wang Mladen Kolar Ziqi Liu Qing Cui Jun Zhou Chaochao Chen FedML 98 11 0 28 Dec 2021
Mime: Mimicking Centralized Stochastic Algorithms in Federated Learning Sai Praneeth Karimireddy Martin Jaggi Satyen Kale M. Mohri Sashank J. Reddi Sebastian U. Stich A. Suresh FedML 66 217 0 08 Aug 2020
SGD for Structured Nonconvex Functions: Learning Rates, Minibatching and Interpolation Robert Mansel Gower Othmane Sebbouh Nicolas Loizou 67 75 0 18 Jun 2020
An Analysis of Constant Step Size SGD in the Non-convex Regime: Asymptotic Normality and Bias Lu Yu Krishnakumar Balasubramanian S. Volgushev Murat A. Erdogdu 73 50 0 14 Jun 2020
A Primer on Zeroth-Order Optimization in Signal Processing and Machine Learning Sijia Liu Pin-Yu Chen B. Kailkhura Gaoyuan Zhang A. Hero III P. Varshney 59 228 0 11 Jun 2020
Randomized Smoothing for Stochastic Optimization John C. Duchi Peter L. Bartlett Martin J. Wainwright 94 282 0 22 Mar 2011
Large Deviations of Vector-valued Martingales in 2-Smooth Normed Spaces A. Juditsky A. Nemirovski 97 97 0 04 Sep 2008