Qualitatively characterizing neural network optimization problems

19 December 2014

Papers citing "Qualitatively characterizing neural network optimization problems"

26 / 125 papers shown

Title
Averaging Weights Leads to Wider Optima and Better Generalization Pavel Izmailov Dmitrii Podoprikhin T. Garipov Dmitry Vetrov A. Wilson FedML MoMe 60 1,621 0 14 Mar 2018
Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs T. Garipov Pavel Izmailov Dmitrii Podoprikhin Dmitry Vetrov A. Wilson UQCV 27 734 0 27 Feb 2018
A Walk with SGD Chen Xing Devansh Arpit Christos Tsirigotis Yoshua Bengio 27 118 0 24 Feb 2018
signSGD: Compressed Optimisation for Non-Convex Problems Jeremy Bernstein Yu Wang Kamyar Azizzadenesheli Anima Anandkumar FedML ODL 44 1,021 0 13 Feb 2018
Visualizing the Loss Landscape of Neural Nets Hao Li Zheng Xu Gavin Taylor Christoph Studer Tom Goldstein 111 1,850 0 28 Dec 2017
Neon2: Finding Local Minima via First-Order Oracles Zeyuan Allen-Zhu Yuanzhi Li 21 130 0 17 Nov 2017
Three Factors Influencing Minima in SGD Stanislaw Jastrzebski Zachary Kenton Devansh Arpit Nicolas Ballas Asja Fischer Yoshua Bengio Amos Storkey 42 457 0 13 Nov 2017
Rethinking generalization requires revisiting old ideas: statistical mechanics approaches and complex learning behavior Charles H. Martin Michael W. Mahoney AI4CE 30 62 0 26 Oct 2017
High-dimensional dynamics of generalization error in neural networks Madhu S. Advani Andrew M. Saxe AI4CE 90 464 0 10 Oct 2017
Natasha 2: Faster Non-Convex Optimization Than SGD Zeyuan Allen-Zhu ODL 28 245 0 29 Aug 2017
Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates L. Smith Nicholay Topin AI4CE 39 519 0 23 Aug 2017
Are Saddles Good Enough for Deep Learning? Adepu Ravi Sankar V. Balasubramanian 43 5 0 07 Jun 2017
The loss surface of deep and wide neural networks Quynh N. Nguyen Matthias Hein ODL 51 283 0 26 Apr 2017
Snapshot Ensembles: Train 1, get M for free Gao Huang Yixuan Li Geoff Pleiss Zhuang Liu J. Hopcroft Kilian Q. Weinberger OOD FedML UQCV 50 935 0 01 Apr 2017
Overcoming Catastrophic Forgetting by Incremental Moment Matching Sang-Woo Lee Jin-Hwa Kim Jaehyun Jun Jung-Woo Ha Byoung-Tak Zhang CLL 30 668 0 24 Mar 2017
An empirical analysis of the optimization of deep network loss surfaces Daniel Jiwoong Im Michael Tao K. Branson ODL 35 61 0 13 Dec 2016
Local minima in training of neural networks G. Swirszcz Wojciech M. Czarnecki Razvan Pascanu ODL 37 73 0 19 Nov 2016
Identity Matters in Deep Learning Moritz Hardt Tengyu Ma OOD 25 398 0 14 Nov 2016
Topology and Geometry of Half-Rectified Network Optimization C. Freeman Joan Bruna 19 233 0 04 Nov 2016
Finding Approximate Local Minima Faster than Gradient Descent Naman Agarwal Zeyuan Allen-Zhu Brian Bullins Elad Hazan Tengyu Ma 43 83 0 03 Nov 2016
On the Expressive Power of Deep Neural Networks M. Raghu Ben Poole Jon M. Kleinberg Surya Ganguli Jascha Narain Sohl-Dickstein 29 778 0 16 Jun 2016
Optimization Methods for Large-Scale Machine Learning Léon Bottou Frank E. Curtis J. Nocedal 105 3,178 0 15 Jun 2016
No bad local minima: Data independent training error guarantees for multilayer neural networks Daniel Soudry Y. Carmon 19 235 0 26 May 2016
Stuck in a What? Adventures in Weight Space Zachary Chase Lipton 23 18 0 23 Feb 2016
Communication-Efficient Learning of Deep Networks from Decentralized Data H. B. McMahan Eider Moore Daniel Ramage S. Hampson Blaise Agüera y Arcas FedML 29 17,071 0 17 Feb 2016
On the Quality of the Initial Basin in Overspecified Neural Networks Itay Safran Ohad Shamir 22 127 0 13 Nov 2015