First-order Methods Almost Always Avoid Saddle Points

First-order Methods Almost Always Avoid Saddle Points

20 October 2017

Ioannis Panageas

Georgios Piliouras

Max Simchowitz

Michael I. Jordan

Benjamin Recht

Papers citing "First-order Methods Almost Always Avoid Saddle Points"

13 / 13 papers shown

Title
Loss Landscape of Shallow ReLU-like Neural Networks: Stationary Points, Saddle Escape, and Network Embedding Zhengqing Wu Berfin Simsek Francois Ged ODL 40 0 0 08 Feb 2024
Stochastic noise can be helpful for variational quantum algorithms Junyu Liu Frederik Wilde A. A. Mele Liang Jiang Jens Eisert Jens Eisert 24 34 0 13 Oct 2022
Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction Kaifeng Lyu Zhiyuan Li Sanjeev Arora FAtt 37 69 0 14 Jun 2022
Training Generative Adversarial Networks with Adaptive Composite Gradient Huiqing Qi Fang Li Shengli Tan Xiangyun Zhang GAN 21 3 0 10 Nov 2021
Safe Reinforcement Learning Using Advantage-Based Intervention Nolan Wagener Byron Boots Ching-An Cheng 29 52 0 16 Jun 2021
On the Impossibility of Global Convergence in Multi-Loss Optimization Alistair Letcher 16 32 0 26 May 2020
Differentiable Game Mechanics Alistair Letcher David Balduzzi S. Racanière James Martens Jakob N. Foerster K. Tuyls T. Graepel 34 79 0 13 May 2019
Optimistic mirror descent in saddle-point problems: Going the extra (gradient) mile P. Mertikopoulos Bruno Lecouat Houssam Zenati Chuan-Sheng Foo V. Chandrasekhar Georgios Piliouras 19 290 0 07 Jul 2018
Defending Against Saddle Point Attack in Byzantine-Robust Distributed Learning Dong Yin Yudong Chen Kannan Ramchandran Peter L. Bartlett FedML 26 97 0 14 Jun 2018
The Global Optimization Geometry of Shallow Linear Neural Networks Zhihui Zhu Daniel Soudry Yonina C. Eldar M. Wakin ODL 18 36 0 13 May 2018
The Mechanics of n-Player Differentiable Games David Balduzzi S. Racanière James Martens Jakob N. Foerster K. Tuyls T. Graepel MLT 16 273 0 15 Feb 2018
Denoising Prior Driven Deep Neural Network for Image Restoration W. Dong Peiyao Wang W. Yin Guangming Shi Fangfang Wu Xiaotong Lu SupR 39 417 0 21 Jan 2018
The Loss Surfaces of Multilayer Networks A. Choromańska Mikael Henaff Michaël Mathieu Gerard Ben Arous Yann LeCun ODL 179 1,185 0 30 Nov 2014