Identifying and attacking the saddle point problem in high-dimensional non-convex optimization

10 June 2014

Papers citing "Identifying and attacking the saddle point problem in high-dimensional non-convex optimization"

50 / 216 papers shown

Title
Impact of classification difficulty on the weight matrices spectra in Deep Learning and application to early-stopping Xuran Meng Jianfeng Yao 27 7 0 26 Nov 2021
Rethinking Generic Camera Models for Deep Single Image Camera Calibration to Recover Rotation and Fisheye Distortion Nobuhiko Wakai Satoshi Sato Yasunori Ishii Takayoshi Yamashita 19 8 0 25 Nov 2021
Inertial Newton Algorithms Avoiding Strict Saddle Points Camille Castera ODL 15 2 0 08 Nov 2021
PAC-Bayesian Learning of Aggregated Binary Activated Neural Networks with Probabilities over Representations Louis Fortier-Dubois Gaël Letarte Benjamin Leblanc Franccois Laviolette Pascal Germain UQCV 17 0 0 28 Oct 2021
Improving Adversarial Robustness for Free with Snapshot Ensemble Yihao Wang AAML UQCV 17 1 0 07 Oct 2021
Boost Neural Networks by Checkpoints Feng Wang Gu-Yeon Wei Qiao Liu Jinxiang Ou Xian Wei Hairong Lv FedML UQCV 27 10 0 03 Oct 2021
Variational learning of quantum ground states on spiking neuromorphic hardware Robert Klassert A. Baumbach Mihai A. Petrovici M. Gärttner 28 7 0 30 Sep 2021
Neural forecasting at scale Philippe Chatigny Shengrui Wang Jean-Marc Patenaude and Boris N. Oreshkin AI4TS 30 1 0 20 Sep 2021
New Q-Newton's method meets Backtracking line search: good convergence guarantee, saddle points avoidance, quadratic rate of convergence, and easy implementation T. Truong 19 5 0 23 Aug 2021
Sparse Bayesian Deep Learning for Dynamic System Identification Hongpeng Zhou Chahine Ibrahim W. Zheng Wei Pan BDL 23 25 0 27 Jul 2021
Activated Gradients for Deep Neural Networks Mei Liu Liangming Chen Xiaohao Du Long Jin Mingsheng Shang ODL AI4CE 35 135 0 09 Jul 2021
Immunization of Pruning Attack in DNN Watermarking Using Constant Weight Code Minoru Kuribayashi Tatsuya Yasui Asad U. Malik N. Funabiki AAML 23 1 0 07 Jul 2021
Control-Oriented Model-Based Reinforcement Learning with Implicit Differentiation Evgenii Nikishin Romina Abachi Rishabh Agarwal Pierre-Luc Bacon OffRL 52 34 0 06 Jun 2021
Escaping Saddle Points with Compressed SGD Dmitrii Avdiukhin G. Yaroslavtsev 22 4 0 21 May 2021
Landscape analysis for shallow neural networks: complete classification of critical points for affine target functions Patrick Cheridito Arnulf Jentzen Florian Rossmannek 24 10 0 19 Mar 2021
Panel semiparametric quantile regression neural network for electricity consumption forecasting Xingcai Zhou Jiangyan Wang AI4TS 14 16 0 01 Mar 2021
Learning Neural Network Subspaces Mitchell Wortsman Maxwell Horton Carlos Guestrin Ali Farhadi Mohammad Rastegari UQCV 27 85 0 20 Feb 2021
Appearance of Random Matrix Theory in Deep Learning Nicholas P. Baskerville Diego Granziol J. Keating 15 11 0 12 Feb 2021
A Deeper Look at the Hessian Eigenspectrum of Deep Neural Networks and its Applications to Regularization Adepu Ravi Sankar Yash Khasbage Rahul Vigneswaran V. Balasubramanian 25 42 0 07 Dec 2020
A Random Matrix Theory Approach to Damping in Deep Learning Diego Granziol Nicholas P. Baskerville AI4CE ODL 29 2 0 15 Nov 2020
Memorizing without overfitting: Bias, variance, and interpolation in over-parameterized models J. Rocks Pankaj Mehta 23 41 0 26 Oct 2020
Softmax Deep Double Deterministic Policy Gradients Ling Pan Qingpeng Cai Longbo Huang 72 86 0 19 Oct 2020
Escaping Saddle Points in Ill-Conditioned Matrix Completion with a Scalable Second Order Method C. Kümmerle C. M. Verdun 19 6 0 07 Sep 2020
A community-powered search of machine learning strategy space to find NMR property prediction models Lars A. Bratholm W. Gerrard Brandon M. Anderson Shaojie Bai Sunghwan Choi ... A. Torrubia Devin Willmott C. Butts David R. Glowacki Kaggle participants 21 16 0 13 Aug 2020
Meta Continual Learning via Dynamic Programming R. Krishnan Prasanna Balaprakash CLL 22 10 0 05 Aug 2020
Reformulation of the No-Free-Lunch Theorem for Entangled Data Sets Kunal Sharma M. Cerezo Zoë Holmes L. Cincio A. Sornborger Patrick J. Coles 18 48 0 09 Jul 2020
Bayesian Sparse learning with preconditioned stochastic gradient MCMC and its applications Yating Wang Wei Deng Guang Lin 18 13 0 29 Jun 2020
An analytic theory of shallow networks dynamics for hinge loss classification Franco Pellegrini Giulio Biroli 35 19 0 19 Jun 2020
Learning Rates as a Function of Batch Size: A Random Matrix Theory Approach to Neural Network Training Diego Granziol S. Zohren Stephen J. Roberts ODL 37 49 0 16 Jun 2020
SPLASH: Learnable Activation Functions for Improving Accuracy and Adversarial Robustness Mohammadamin Tavakoli Forest Agostinelli Pierre Baldi AAML FAtt 36 39 0 16 Jun 2020
On the Promise of the Stochastic Generalized Gauss-Newton Method for Training DNNs Matilde Gargiani Andrea Zanelli Moritz Diehl Frank Hutter ODL 4 18 0 03 Jun 2020
Review of Mathematical frameworks for Fairness in Machine Learning E. del Barrio Paula Gordaliza Jean-Michel Loubes FaML FedML 15 38 0 26 May 2020
DENS-ECG: A Deep Learning Approach for ECG Signal Delineation A. Peimankar S. Puthusserypady 36 119 0 18 May 2020
Symmetry & critical points for a model shallow neural network Yossi Arjevani M. Field 34 13 0 23 Mar 2020
Critical Point-Finding Methods Reveal Gradient-Flat Regions of Deep Network Losses Charles G. Frye James B. Simon Neha S. Wadia A. Ligeralde M. DeWeese K. Bouchard ODL 16 2 0 23 Mar 2020
Block Layer Decomposition schemes for training Deep Neural Networks L. Palagi R. Seccia 15 5 0 18 Mar 2020
On the Effectiveness of Mitigating Data Poisoning Attacks with Gradient Shaping Sanghyun Hong Varun Chandrasekaran Yigitcan Kaya Tudor Dumitras Nicolas Papernot AAML 28 136 0 26 Feb 2020
The Break-Even Point on Optimization Trajectories of Deep Neural Networks Stanislaw Jastrzebski Maciej Szymczak Stanislav Fort Devansh Arpit Jacek Tabor Kyunghyun Cho Krzysztof J. Geras 50 154 0 21 Feb 2020
$Depth Descent Synchronization in $\mathrm{SO}(D)$$ Depth Descent Synchronization in $\mathrm{SO}(D)$ Tyler Maunu Gilad Lerman MDE 37 2 0 13 Feb 2020
Low Rank Saddle Free Newton: A Scalable Method for Stochastic Nonconvex Optimization Thomas O'Leary-Roseberry Nick Alger Omar Ghattas ODL 37 9 0 07 Feb 2020
Optimization for deep learning: theory and algorithms Ruoyu Sun ODL 25 168 0 19 Dec 2019
Information-Theoretic Local Minima Characterization and Regularization Zhiwei Jia Hao Su 27 19 0 19 Nov 2019
Geometry of learning neural quantum states Chae-Yeun Park M. Kastoryano 26 60 0 24 Oct 2019
Hidden Unit Specialization in Layered Neural Networks: ReLU vs. Sigmoidal Activation Elisa Oostwal Michiel Straat Michael Biehl MLT 56 55 0 16 Oct 2019
Quantum algorithm for finding the negative curvature direction in non-convex optimization Kaining Zhang Min-hsiu Hsieh Liu Liu Dacheng Tao 15 3 0 17 Sep 2019
diffGrad: An Optimization Method for Convolutional Neural Networks S. Dubey Soumendu Chakraborty Swalpa Kumar Roy Snehasis Mukherjee S. Singh B. B. Chaudhuri ODL 97 184 0 12 Sep 2019
Solving Continual Combinatorial Selection via Deep Reinforcement Learning Hyungseok Song Hyeryung Jang H. Tran Se-eun Yoon Kyunghwan Son Donggyu Yun Hyoju Chung Yung Yi 18 10 0 09 Sep 2019
Distributed Gradient Descent: Nonconvergence to Saddle Points and the Stable-Manifold Theorem Brian Swenson Ryan W. Murray H. Vincent Poor S. Kar 26 14 0 07 Aug 2019
Weight-space symmetry in deep networks gives rise to permutation saddles, connected by equal-loss valleys across the loss landscape Johanni Brea Berfin Simsek Bernd Illing W. Gerstner 23 55 0 05 Jul 2019
Global Convergence of Policy Gradient Methods to (Almost) Locally Optimal Policies Kaipeng Zhang Alec Koppel Haoqi Zhu Tamer Basar 44 186 0 19 Jun 2019