ResearchTrend.AI
FOSI: Hybrid First and Second Order Optimization
v1v2v3v4 (latest)

FOSI: Hybrid First and Second Order Optimization

Hadar Sivan
Moshe Gabel
Assaf Schuster
    ODL
ArXiv (abs)PDFHTML

Papers citing "FOSI: Hybrid First and Second Order Optimization"

7 / 7 papers shown
Title
KAISA: An Adaptive Second-Order Optimizer Framework for Deep Neural Networks
KAISA: An Adaptive Second-Order Optimizer Framework for Deep Neural Networks
J. G. Pauloski
Qi Huang
Lei Huang
Shivaram Venkataraman
Kyle Chard
Ian Foster
Zhao-jie Zhang
60
29
0
04 Jul 2021
Towards Theoretically Understanding Why SGD Generalizes Better Than ADAM in Deep Learning
Towards Theoretically Understanding Why SGD Generalizes Better Than ADAM in Deep Learning
Pan Zhou
Jiashi Feng
Chao Ma
Caiming Xiong
Guosheng Lin
E. Weinan
86
234
0
12 Oct 2020
ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning
ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning
Z. Yao
A. Gholami
Sheng Shen
Mustafa Mustafa
Kurt Keutzer
Michael W. Mahoney
ODL
108
285
0
01 Jun 2020
Shampoo: Preconditioned Stochastic Tensor Optimization
Shampoo: Preconditioned Stochastic Tensor Optimization
Vineet Gupta
Tomer Koren
Y. Singer
ODL
95
226
0
26 Feb 2018
The Marginal Value of Adaptive Gradient Methods in Machine Learning
The Marginal Value of Adaptive Gradient Methods in Machine Learning
Ashia Wilson
Rebecca Roelofs
Mitchell Stern
Nathan Srebro
Benjamin Recht
ODL
71
1,032
0
23 May 2017
Sub-sampled Newton Methods with Non-uniform Sampling
Sub-sampled Newton Methods with Non-uniform Sampling
Peng Xu
Jiyan Yang
Farbod Roosta-Khorasani
Christopher Ré
Michael W. Mahoney
70
117
0
02 Jul 2016
Optimizing Neural Networks with Kronecker-factored Approximate Curvature
Optimizing Neural Networks with Kronecker-factored Approximate Curvature
James Martens
Roger C. Grosse
ODL
104
1,023
0
19 Mar 2015