Stochastic Cubic Regularization for Fast Nonconvex Optimization

8 November 2017

Papers citing "Stochastic Cubic Regularization for Fast Nonconvex Optimization"

32 / 32 papers shown

Title
SAPPHIRE: Preconditioned Stochastic Variance Reduction for Faster Large-Scale Statistical Learning Jingruo Sun Zachary Frangella Madeleine Udell 36 0 0 28 Jan 2025
Second-Order Min-Max Optimization with Lazy Hessians Lesi Chen Chengchang Liu Jingzhao Zhang 46 1 0 12 Oct 2024
Cubic regularized subspace Newton for non-convex optimization Jim Zhao Aurelien Lucchi N. Doikov 33 5 0 24 Jun 2024
On Newton's Method to Unlearn Neural Networks Nhung Bui Xinyang Lu Rachael Hwee Ling Sim See-Kiong Ng Bryan Kian Hsiang Low MU 43 2 0 20 Jun 2024
Accelerated Stochastic Min-Max Optimization Based on Bias-corrected Momentum H. Cai Sulaiman A. Alghunaim Ali H.Sayed 52 1 0 18 Jun 2024
Random Scaling and Momentum for Non-smooth Non-convex Optimization Qinzi Zhang Ashok Cutkosky 43 4 0 16 May 2024
Efficiently Escaping Saddle Points for Non-Convex Policy Optimization Sadegh Khorasani Saber Salehkaleybar Negar Kiyavash Niao He Matthias Grossglauser 29 1 0 15 Nov 2023
Unified Convergence Theory of Stochastic and Variance-Reduced Cubic Newton Methods El Mahdi Chayti N. Doikov Martin Jaggi ODL 27 6 0 23 Feb 2023
Faster Riemannian Newton-type Optimization by Subsampling and Cubic Regularization Yian Deng Tingting Mu 26 1 0 22 Feb 2023
Stochastic Dimension-reduced Second-order Methods for Policy Optimization Jinsong Liu Chen Xie Qinwen Deng Dongdong Ge Yi-Li Ye 32 1 0 28 Jan 2023
SP2: A Second Order Stochastic Polyak Method Shuang Li W. Swartworth Martin Takávc Deanna Needell Robert Mansel Gower 29 13 0 17 Jul 2022
Stochastic Variance-Reduced Newton: Accelerating Finite-Sum Minimization with Large Batches Michal Derezinski 55 5 0 06 Jun 2022
Augmented Newton Method for Optimization: Global Linear Rate and Momentum Interpretation M. Morshed ODL 24 1 0 23 May 2022
Tackling benign nonconvexity with smoothing and stochastic gradients Harsh Vardhan Sebastian U. Stich 31 8 0 18 Feb 2022
Escape saddle points by a simple gradient-descent based algorithm Chenyi Zhang Tongyang Li ODL 31 15 0 28 Nov 2021
Escaping Saddle Points with Compressed SGD Dmitrii Avdiukhin G. Yaroslavtsev 22 4 0 21 May 2021
HEBO Pushing The Limits of Sample-Efficient Hyperparameter Optimisation Alexander I. Cowen-Rivers Wenlong Lyu Rasul Tutunov Zhi Wang Antoine Grosnit ... A. Maraval Hao Jianye Jun Wang Jan Peters H. Ammar 27 74 0 07 Dec 2020
Optimization for Supervised Machine Learning: Randomized Algorithms for Data and Parameters Filip Hanzely 39 0 0 26 Aug 2020
Second-Order Information in Non-Convex Stochastic Optimization: Power and Limitations Yossi Arjevani Y. Carmon John C. Duchi Dylan J. Foster Ayush Sekhari Karthik Sridharan 90 53 0 24 Jun 2020
An Analysis of Constant Step Size SGD in the Non-convex Regime: Asymptotic Normality and Bias Lu Yu Krishnakumar Balasubramanian S. Volgushev Murat A. Erdogdu 35 50 0 14 Jun 2020
Adaptivity of Stochastic Gradient Methods for Nonconvex Optimization Samuel Horváth Lihua Lei Peter Richtárik Michael I. Jordan 57 30 0 13 Feb 2020
Momentum Improves Normalized SGD Ashok Cutkosky Harsh Mehta ODL 18 118 0 09 Feb 2020
Adaptive Stochastic Optimization Frank E. Curtis K. Scheinberg ODL 16 29 0 18 Jan 2020
Stabilized SVRG: Simple Variance Reduction for Nonconvex Optimization Rong Ge Zhize Li Weiyao Wang Xiang Wang 19 34 0 01 May 2019
Negative eigenvalues of the Hessian in deep neural networks Guillaume Alain Nicolas Le Roux Pierre-Antoine Manzagol 21 42 0 06 Feb 2019
Optimal Adaptive and Accelerated Stochastic Gradient Descent Qi Deng Yi Cheng Guanghui Lan 16 8 0 01 Oct 2018
Convergence of Cubic Regularization for Nonconvex Optimization under KL Property Yi Zhou Zhe Wang Yingbin Liang 24 23 0 22 Aug 2018
SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimator Cong Fang C. J. Li Zhouchen Lin Tong Zhang 50 571 0 04 Jul 2018
Stochastic Variance-Reduced Cubic Regularization for Nonconvex Optimization Zhe Wang Yi Zhou Yingbin Liang Guanghui Lan 35 47 0 20 Feb 2018
Natasha 2: Faster Non-Convex Optimization Than SGD Zeyuan Allen-Zhu ODL 28 245 0 29 Aug 2017
Asymptotic Optimality in Stochastic Optimization John C. Duchi Feng Ruan 15 61 0 16 Dec 2016
The Loss Surfaces of Multilayer Networks A. Choromańska Mikael Henaff Michaël Mathieu Gerard Ben Arous Yann LeCun ODL 186 1,186 0 30 Nov 2014