Small nonlinearities in activation functions create bad local minima in
neural networks

Small nonlinearities in activation functions create bad local minima in neural networks

10 February 2018

Papers citing "Small nonlinearities in activation functions create bad local minima in neural networks"

18 / 18 papers shown

Title
Read the Signs: Towards Invariance to Gradient Descent's Hyperparameter Initialization Davood Wadi M. Fredette S. Sénécal ODL AI4CE 8 0 0 24 Jan 2023
When Expressivity Meets Trainability: Fewer than $n$ Neurons Can Work Jiawei Zhang Yushun Zhang Mingyi Hong Ruoyu Sun Zhi-Quan Luo 29 10 0 21 Oct 2022
Exact Solutions of a Deep Linear Network Liu Ziyin Botao Li Xiangmin Meng ODL 19 21 0 10 Feb 2022
Exponentially Many Local Minima in Quantum Neural Networks Xuchen You Xiaodi Wu 72 51 0 06 Oct 2021
A Geometric Analysis of Neural Collapse with Unconstrained Features Zhihui Zhu Tianyu Ding Jinxin Zhou Xiao Li Chong You Jeremias Sulam Qing Qu 38 196 0 06 May 2021
Review: Deep Learning in Electron Microscopy Jeffrey M. Ede 34 79 0 17 Sep 2020
A Mean-field Analysis of Deep ResNet and Beyond: Towards Provable Optimization Via Overparameterization From Depth Yiping Lu Chao Ma Yulong Lu Jianfeng Lu Lexing Ying MLT 39 78 0 11 Mar 2020
The Usual Suspects? Reassessing Blame for VAE Posterior Collapse Bin Dai Ziyu Wang David Wipf DRL 24 75 0 23 Dec 2019
Optimization for deep learning: theory and algorithms Ruoyu Sun ODL 25 168 0 19 Dec 2019
Hessian based analysis of SGD for Deep Nets: Dynamics and Generalization Xinyan Li Qilong Gu Yingxue Zhou Tiancong Chen A. Banerjee ODL 42 51 0 24 Jul 2019
Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks Sanjeev Arora S. Du Wei Hu Zhiyuan Li Ruosong Wang MLT 55 961 0 24 Jan 2019
Width Provably Matters in Optimization for Deep Linear Neural Networks S. Du Wei Hu 21 94 0 24 Jan 2019
Non-attracting Regions of Local Minima in Deep and Wide Neural Networks Henning Petzka C. Sminchisescu 29 9 0 16 Dec 2018
Subgradient Descent Learns Orthogonal Dictionaries Yu Bai Qijia Jiang Ju Sun 20 51 0 25 Oct 2018
Small ReLU networks are powerful memorizers: a tight analysis of memorization capacity Chulhee Yun S. Sra Ali Jadbabaie 26 117 0 17 Oct 2018
Learning ReLU Networks on Linearly Separable Data: Algorithm, Optimality, and Generalization G. Wang G. Giannakis Jie Chen MLT 24 131 0 14 Aug 2018
Global optimality conditions for deep neural networks Chulhee Yun S. Sra Ali Jadbabaie 128 117 0 08 Jul 2017
The Loss Surfaces of Multilayer Networks A. Choromańska Mikael Henaff Michaël Mathieu Gerard Ben Arous Yann LeCun ODL 183 1,185 0 30 Nov 2014