ResearchTrend.AI
CProp: Adaptive Learning Rate Scaling from Past Gradient Conformity

CProp: Adaptive Learning Rate Scaling from Past Gradient Conformity

Konpat Preechakul
B. Kijsirikul
    ODL
ArXivPDFHTML

Papers citing "CProp: Adaptive Learning Rate Scaling from Past Gradient Conformity"

3 / 3 papers shown
Title
Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers
Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers
Robin M. Schmidt
Frank Schneider
Philipp Hennig
ODL
42
162
0
03 Jul 2020
A disciplined approach to neural network hyper-parameters: Part 1 -- learning rate, batch size, momentum, and weight decay
A disciplined approach to neural network hyper-parameters: Part 1 -- learning rate, batch size, momentum, and weight decay
L. Smith
208
1,020
0
26 Mar 2018
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima
N. Keskar
Dheevatsa Mudigere
J. Nocedal
M. Smelyanskiy
P. T. P. Tang
ODL
310
2,896
0
15 Sep 2016