Fast large-scale optimization by unifying stochastic gradient and quasi-Newton methods

9 November 2013

Jascha Narain Sohl-Dickstein

Papers citing "Fast large-scale optimization by unifying stochastic gradient and quasi-Newton methods"

20 / 20 papers shown

Title
FedSSO: A Federated Server-Side Second-Order Optimization Algorithm Xinteng Ma Renyi Bao Jinpeng Jiang Yang Liu Arthur Jiang Junhua Yan Xin Liu Zhisong Pan FedML 32 6 0 20 Jun 2022
KOALA: A Kalman Optimization Algorithm with Loss Adaptivity A. Davtyan Sepehr Sameni L. Cerkezi Givi Meishvili Adam Bielski Paolo Favaro ODL 53 2 0 07 Jul 2021
Whitening and second order optimization both make information in the dataset unusable during training, and can reduce or prevent generalization Neha S. Wadia Daniel Duckworth S. Schoenholz Ethan Dyer Jascha Narain Sohl-Dickstein 24 13 0 17 Aug 2020
Optimization of neural networks via finite-value quantum fluctuations Masayuki Ohzeki Shuntaro Okada Masayoshi Terabe S. Taguchi 13 21 0 01 Jul 2018
Algorithms for solving optimization problems arising from deep neural net models: smooth problems Vyacheslav Kungurtsev Tomás Pevný 16 6 0 30 Jun 2018
Analysis of DAWNBench, a Time-to-Accuracy Machine Learning Performance Benchmark Cody Coleman Daniel Kang Deepak Narayanan Luigi Nardi Tian Zhao Jian Zhang Peter Bailis K. Olukotun Christopher Ré Matei A. Zaharia 13 117 0 04 Jun 2018
Stochastic Training of Neural Networks via Successive Convex Approximations Simone Scardapane P. Di Lorenzo 16 9 0 15 Jun 2017
Proximal Backpropagation Thomas Frerix Thomas Möllenhoff Michael Möller Daniel Cremers 21 31 0 14 Jun 2017
DeepGaze II: Reading fixations from deep features trained on object recognition Matthias Kümmerer Thomas S. A. Wallis Matthias Bethge 18 287 0 05 Oct 2016
A Kronecker-factored approximate Fisher matrix for convolution layers Roger C. Grosse James Martens ODL 26 257 0 03 Feb 2016
A Linearly-Convergent Stochastic L-BFGS Algorithm Philipp Moritz Robert Nishihara Michael I. Jordan ODL 27 232 0 09 Aug 2015
Tensor machines for learning target-specific polynomial features Jiyan Yang Alex Gittens 21 8 0 07 Apr 2015
Deep Unsupervised Learning using Nonequilibrium Thermodynamics Jascha Narain Sohl-Dickstein Eric A. Weiss Niru Maheswaranathan Surya Ganguli SyDa DiffM 59 6,592 0 12 Mar 2015
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 22 148,823 0 22 Dec 2014
Discovering Hidden Factors of Variation in Deep Networks Brian Cheung J. Livezey Arjun K. Bansal Bruno A. Olshausen DRL 22 193 0 20 Dec 2014
Deep Gaze I: Boosting Saliency Prediction with Feature Maps Trained on ImageNet Matthias Kümmerer Lucas Theis Matthias Bethge FAtt 27 406 0 04 Nov 2014
Identifying and attacking the saddle point problem in high-dimensional non-convex optimization Yann N. Dauphin Razvan Pascanu Çağlar Gülçehre Kyunghyun Cho Surya Ganguli Yoshua Bengio ODL 29 1,376 0 10 Jun 2014
Incremental Majorization-Minimization Optimization with Application to Large-Scale Machine Learning Julien Mairal 76 317 0 18 Feb 2014
Minimizing Finite Sums with the Stochastic Average Gradient Mark W. Schmidt Nicolas Le Roux Francis R. Bach 52 1,243 0 10 Sep 2013
Improving neural networks by preventing co-adaptation of feature detectors Geoffrey E. Hinton Nitish Srivastava A. Krizhevsky Ilya Sutskever Ruslan Salakhutdinov VLM 266 7,636 0 03 Jul 2012