First-order Methods Almost Always Avoid Saddle Points

First-order Methods Almost Always Avoid Saddle Points

20 October 2017

Ioannis Panageas

Georgios Piliouras

Max Simchowitz

Michael I. Jordan

Benjamin Recht

Papers citing "First-order Methods Almost Always Avoid Saddle Points"

11 / 11 papers shown

Title
Loss Landscape of Shallow ReLU-like Neural Networks: Stationary Points, Saddle Escape, and Network Embedding Zhengqing Wu Berfin Simsek Francois Ged ODL 40 0 0 08 Feb 2024
Stochastic noise can be helpful for variational quantum algorithms Junyu Liu Frederik Wilde A. A. Mele Liang Jiang Jens Eisert Jens Eisert 16 34 0 13 Oct 2022
Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction Kaifeng Lyu Zhiyuan Li Sanjeev Arora FAtt 37 69 0 14 Jun 2022
Training Generative Adversarial Networks with Adaptive Composite Gradient Huiqing Qi Fang Li Shengli Tan Xiangyun Zhang GAN 21 3 0 10 Nov 2021
Safe Reinforcement Learning Using Advantage-Based Intervention Nolan Wagener Byron Boots Ching-An Cheng 27 52 0 16 Jun 2021
On the Impossibility of Global Convergence in Multi-Loss Optimization Alistair Letcher 13 32 0 26 May 2020
Differentiable Game Mechanics Alistair Letcher David Balduzzi S. Racanière James Martens Jakob N. Foerster K. Tuyls T. Graepel 29 79 0 13 May 2019
Defending Against Saddle Point Attack in Byzantine-Robust Distributed Learning Dong Yin Yudong Chen K. Ramchandran Peter L. Bartlett FedML 24 97 0 14 Jun 2018
The Global Optimization Geometry of Shallow Linear Neural Networks Zhihui Zhu Daniel Soudry Yonina C. Eldar M. Wakin ODL 16 36 0 13 May 2018
The Mechanics of n-Player Differentiable Games David Balduzzi S. Racanière James Martens Jakob N. Foerster K. Tuyls T. Graepel MLT 16 273 0 15 Feb 2018
The Loss Surfaces of Multilayer Networks A. Choromańska Mikael Henaff Michaël Mathieu Gerard Ben Arous Yann LeCun ODL 179 1,185 0 30 Nov 2014