Stochasticity helps to navigate rough landscapes: comparing
gradient-descent-based algorithms in the phase retrieval problem

Stochasticity helps to navigate rough landscapes: comparing gradient-descent-based algorithms in the phase retrieval problem

8 March 2021

Francesca Mignacco

Pierfrancesco Urbani

Lenka Zdeborová

Papers citing "Stochasticity helps to navigate rough landscapes: comparing gradient-descent-based algorithms in the phase retrieval problem"

19 / 19 papers shown

Title
Bilinear Sequence Regression: A Model for Learning from Long Sequences of High-dimensional Tokens Vittorio Erba Emanuele Troiani Luca Biggio Antoine Maillard Lenka Zdeborová 91 1 0 24 Oct 2024
Optimal Combination of Linear and Spectral Estimators for Generalized Linear Models Marco Mondelli Christos Thrampoulidis R. Venkataramanan 35 16 0 07 Aug 2020
Optimization and Generalization of Shallow Neural Networks with Quadratic Activation Functions Stefano Sarao Mannelli Eric Vanden-Eijnden Lenka Zdeborová AI4CE 19 46 0 27 Jun 2020
Shape Matters: Understanding the Implicit Bias of the Noise Covariance Jeff Z. HaoChen Colin Wei Jason D. Lee Tengyu Ma 70 94 0 15 Jun 2020
Complex Dynamics in Simple Neural Networks: Understanding Gradient Flow in Phase Retrieval Stefano Sarao Mannelli Giulio Biroli C. Cammarota Florent Krzakala Pierfrancesco Urbani Lenka Zdeborová 20 28 0 12 Jun 2020
Dynamical mean-field theory for stochastic gradient descent in Gaussian mixture classification Francesca Mignacco Florent Krzakala Pierfrancesco Urbani Lenka Zdeborová MLT 34 67 0 10 Jun 2020
Generalisation error in learning with random features and the hidden manifold model Federica Gerace Bruno Loureiro Florent Krzakala M. Mézard Lenka Zdeborová 43 168 0 21 Feb 2020
Poly-time universality and limitations of deep learning Emmanuel Abbe Colin Sandon 16 23 0 07 Jan 2020
Deep Double Descent: Where Bigger Models and More Data Hurt Preetum Nakkiran Gal Kaplun Yamini Bansal Tristan Yang Boaz Barak Ilya Sutskever 77 925 0 04 Dec 2019
Online Stochastic Gradient Descent with Arbitrary Initialization Solves Non-smooth, Non-convex Phase Retrieval Yan Shuo Tan Roman Vershynin 39 35 0 28 Oct 2019
Bad Global Minima Exist and SGD Can Reach Them Shengchao Liu Dimitris Papailiopoulos D. Achlioptas 38 80 0 06 Jun 2019
A Tail-Index Analysis of Stochastic Gradient Noise in Deep Neural Networks Umut Simsekli Levent Sagun Mert Gurbuzbalaban 62 241 0 18 Jan 2019
Gradient Descent with Random Initialization: Fast Global Convergence for Nonconvex Phase Retrieval Yuxin Chen Yuejie Chi Jianqing Fan Cong Ma 37 235 0 21 Mar 2018
$Optimization-based AMP for Phase Retrieval: The Impact of Initialization and $\ell_2$-regularization$ Optimization-based AMP for Phase Retrieval: The Impact of Initialization and $\ell_2$ -regularization Junjie Ma Ji Xu A. Maleki 58 53 0 03 Jan 2018
Three Factors Influencing Minima in SGD Stanislaw Jastrzebski Zachary Kenton Devansh Arpit Nicolas Ballas Asja Fischer Yoshua Bengio Amos Storkey 53 458 0 13 Nov 2017
Fundamental Limits of Weak Recovery with Applications to Phase Retrieval Marco Mondelli Andrea Montanari 49 119 0 20 Aug 2017
Optimal Errors and Phase Transitions in High-Dimensional Generalized Linear Models Jean Barbier Florent Krzakala N. Macris Léo Miolane Lenka Zdeborová 55 262 0 10 Aug 2017
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 335 2,913 0 15 Sep 2016
Stochastic modified equations and adaptive stochastic gradient algorithms Qianxiao Li Cheng Tai E. Weinan 47 282 0 19 Nov 2015