Second-Order Optimization for Non-Convex Machine Learning: An Empirical Study

25 August 2017

Peng Xu

Farbod Roosta-Khorasani

Michael W. Mahoney

ODL

ArXiv PDF HTML

Papers citing "Second-Order Optimization for Non-Convex Machine Learning: An Empirical Study"

20 / 20 papers shown

Title
Online Control-Informed Learning Zihao Liang Tianyu Zhou Zehui Lu Shaoshuai Mou 33 1 0 04 Oct 2024
Newton Method-based Subspace Support Vector Data Description Fahad Sohrab Firas Laakom Moncef Gabbouj 19 5 0 25 Sep 2023
PyPop7: A Pure-Python Library for Population-Based Black-Box Optimization Qiqi Duan Guochen Zhou Chang Shao Zhuowei Wang Mingyang Feng Yuwei Huang Yajing Tan Yijun Yang Qi Zhao Yuhui Shi 34 5 0 12 Dec 2022
HessianFR: An Efficient Hessian-based Follow-the-Ridge Algorithm for Minimax Optimization Yihang Gao Huafeng Liu Michael K. Ng Mingjie Zhou 25 2 0 23 May 2022
Hessian Averaging in Stochastic Newton Methods Achieves Superlinear Convergence Sen Na Michal Derezinski Michael W. Mahoney 27 16 0 20 Apr 2022
Distributed Learning With Sparsified Gradient Differences Yicheng Chen Rick S. Blum Martin Takáč Brian M. Sadler 31 15 0 05 Feb 2022
Large-Scale Deep Learning Optimizations: A Comprehensive Survey Xiaoxin He Fuzhao Xue Xiaozhe Ren Yang You 30 14 0 01 Nov 2021
slimTrain -- A Stochastic Approximation Method for Training Separable Deep Neural Networks Elizabeth Newman Julianne Chung Matthias Chung Lars Ruthotto 47 6 0 28 Sep 2021
Nonlinear Least Squares for Large-Scale Machine Learning using Stochastic Jacobian Estimates Johannes J Brust 8 2 0 12 Jul 2021
SHINE: SHaring the INverse Estimate from the forward pass for bi-level optimization and implicit models Zaccharie Ramzi Florian Mannel Shaojie Bai Jean-Luc Starck P. Ciuciu Thomas Moreau 31 28 0 01 Jun 2021
When Does Preconditioning Help or Hurt Generalization? S. Amari Jimmy Ba Roger C. Grosse Xuechen Li Atsushi Nitanda Taiji Suzuki Denny Wu Ji Xu 36 32 0 18 Jun 2020
SONIA: A Symmetric Blockwise Truncated Optimization Algorithm Majid Jahani M. Nazari R. Tappenden A. Berahas Martin Takávc ODL 11 10 0 06 Jun 2020
ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning Z. Yao A. Gholami Sheng Shen Mustafa Mustafa Kurt Keutzer Michael W. Mahoney ODL 39 275 0 01 Jun 2020
An Inertial Newton Algorithm for Deep Learning Camille Castera Jérôme Bolte Cédric Févotte Edouard Pauwels PINN ODL 20 62 0 29 May 2019
Quasi-Newton Methods for Machine Learning: Forget the Past, Just Sample A. Berahas Majid Jahani Peter Richtárik Martin Takávc 24 40 0 28 Jan 2019
Parameter Re-Initialization through Cyclical Batch Size Schedules Norman Mu Z. Yao A. Gholami Kurt Keutzer Michael W. Mahoney ODL 30 8 0 04 Dec 2018
Structured Local Optima in Sparse Blind Deconvolution Yuqian Zhang Han-Wen Kuo John N. Wright 26 56 0 01 Jun 2018
GPU Accelerated Sub-Sampled Newton's Method Sudhir B. Kylasa Farbod Roosta-Khorasani Michael W. Mahoney A. Grama ODL 26 8 0 26 Feb 2018
GIANT: Globally Improved Approximate Newton Method for Distributed Optimization Shusen Wang Farbod Roosta-Khorasani Peng Xu Michael W. Mahoney 36 127 0 11 Sep 2017
The Loss Surfaces of Multilayer Networks A. Choromańska Mikael Henaff Michaël Mathieu Gerard Ben Arous Yann LeCun ODL 183 1,185 0 30 Nov 2014