Learning ReLUs via Gradient Descent

10 May 2017

Papers citing "Learning ReLUs via Gradient Descent"

49 / 49 papers shown

Title
Gradient-Based Feature Learning under Structured Data Alireza Mousavi-Hosseini Denny Wu Taiji Suzuki Murat A. Erdogdu MLT 37 18 0 07 Sep 2023
On Single Index Models beyond Gaussian Data Joan Bruna Loucas Pillaud-Vivien Aaron Zweig 18 10 0 28 Jul 2023
A faster and simpler algorithm for learning shallow networks Sitan Chen Shyam Narayanan 41 7 0 24 Jul 2023
Smoothing the Landscape Boosts the Signal for SGD: Optimal Sample Complexity for Learning Single Index Models Alexandru Damian Eshaan Nichani Rong Ge Jason D. Lee MLT 42 33 0 18 May 2023
Provable Guarantees for Nonlinear Feature Learning in Three-Layer Neural Networks Eshaan Nichani Alexandru Damian Jason D. Lee MLT 44 13 0 11 May 2023
Over-Parameterization Exponentially Slows Down Gradient Descent for Learning a Single Neuron Weihang Xu S. Du 37 16 0 20 Feb 2023
Near-Optimal Cryptographic Hardness of Agnostically Learning Halfspaces and ReLU Regression under Gaussian Marginals Ilias Diakonikolas D. Kane Lisheng Ren 17 26 0 13 Feb 2023
Learning Single-Index Models with Shallow Neural Networks A. Bietti Joan Bruna Clayton Sanford M. Song 170 68 0 27 Oct 2022
SQ Lower Bounds for Learning Single Neurons with Massart Noise Ilias Diakonikolas D. Kane Lisheng Ren Yuxin Sun 25 6 0 18 Oct 2022
Towards Theoretically Inspired Neural Initialization Optimization Yibo Yang Hong Wang Haobo Yuan Zhouchen Lin 26 9 0 12 Oct 2022
Neural Networks can Learn Representations with Gradient Descent Alexandru Damian Jason D. Lee Mahdi Soltanolkotabi SSL MLT 25 114 0 30 Jun 2022
The Mechanism of Prediction Head in Non-contrastive Self-supervised Learning Zixin Wen Yuanzhi Li SSL 32 34 0 12 May 2022
ReLU Regression with Massart Noise Ilias Diakonikolas Jongho Park Christos Tzamos 56 11 0 10 Sep 2021
Toward Understanding the Feature Learning Process of Self-supervised Contrastive Learning Zixin Wen Yuanzhi Li SSL MLT 32 131 0 31 May 2021
Learning Graph Neural Networks with Approximate Gradient Descent Qunwei Li Shaofeng Zou Leon Wenliang Zhong GNN 32 1 0 07 Dec 2020
Towards a Mathematical Understanding of Neural Network-Based Machine Learning: what we know and what we don't E. Weinan Chao Ma Stephan Wojtowytsch Lei Wu AI4CE 22 133 0 22 Sep 2020
Generalized Leverage Score Sampling for Neural Networks J. Lee Ruoqi Shen Zhao Song Mengdi Wang Zheng Yu 21 42 0 21 Sep 2020
Nonparametric Learning of Two-Layer ReLU Residual Units Zhunxuan Wang Linyun He Chunchuan Lyu Shay B. Cohen MLT OffRL 33 1 0 17 Aug 2020
From Boltzmann Machines to Neural Networks and Back Again Surbhi Goel Adam R. Klivans Frederic Koehler 19 5 0 25 Jul 2020
Near-Optimal SQ Lower Bounds for Agnostically Learning Halfspaces and ReLUs under Gaussian Marginals Ilias Diakonikolas D. Kane Nikos Zarifis 19 66 0 29 Jun 2020
Approximation Schemes for ReLU Regression Ilias Diakonikolas Surbhi Goel Sushrut Karmalkar Adam R. Klivans Mahdi Soltanolkotabi 18 51 0 26 May 2020
Denoising and Regularization via Exploiting the Structural Bias of Convolutional Generators Reinhard Heckel Mahdi Soltanolkotabi DiffM 35 81 0 31 Oct 2019
The Local Elasticity of Neural Networks Hangfeng He Weijie J. Su 40 44 0 15 Oct 2019
Learning Distributions Generated by One-Layer ReLU Networks Shanshan Wu A. Dimakis Sujay Sanghavi 11 22 0 04 Sep 2019
Gradient Descent can Learn Less Over-parameterized Two-layer Neural Networks on Classification Problems Atsushi Nitanda Geoffrey Chinot Taiji Suzuki MLT 16 33 0 23 May 2019
A Selective Overview of Deep Learning Jianqing Fan Cong Ma Yiqiao Zhong BDL VLM 38 136 0 10 Apr 2019
Every Local Minimum Value is the Global Minimum Value of Induced Model in Non-convex Machine Learning Kenji Kawaguchi Jiaoyang Huang L. Kaelbling AAML 24 18 0 07 Apr 2019
On the Power and Limitations of Random Features for Understanding Neural Networks Gilad Yehudai Ohad Shamir MLT 26 181 0 01 Apr 2019
Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks Sanjeev Arora S. Du Wei Hu Zhiyuan Li Ruosong Wang MLT 55 961 0 24 Jan 2019
Width Provably Matters in Optimization for Deep Linear Neural Networks S. Du Wei Hu 21 94 0 24 Jan 2019
Convex Relaxations of Convolutional Neural Nets Burak Bartan Mert Pilanci 20 5 0 31 Dec 2018
Gradient Descent Finds Global Minima of Deep Neural Networks S. Du J. Lee Haochuan Li Liwei Wang Masayoshi Tomizuka ODL 44 1,122 0 09 Nov 2018
On the Convergence Rate of Training Recurrent Neural Networks Zeyuan Allen-Zhu Yuanzhi Li Zhao Song 23 191 0 29 Oct 2018
Subgradient Descent Learns Orthogonal Dictionaries Yu Bai Qijia Jiang Ju Sun 20 51 0 25 Oct 2018
Small ReLU networks are powerful memorizers: a tight analysis of memorization capacity Chulhee Yun S. Sra Ali Jadbabaie 26 117 0 17 Oct 2018
Learning Two-layer Neural Networks with Symmetric Inputs Rong Ge Rohith Kuditipudi Zhize Li Xiang Wang OOD MLT 36 57 0 16 Oct 2018
Gradient Descent Provably Optimizes Over-parameterized Neural Networks S. Du Xiyu Zhai Barnabás Póczós Aarti Singh MLT ODL 53 1,252 0 04 Oct 2018
On the loss landscape of a class of deep neural networks with no bad local valleys Quynh N. Nguyen Mahesh Chandra Mukkamala Matthias Hein 16 87 0 27 Sep 2018
Learning ReLU Networks on Linearly Separable Data: Algorithm, Optimality, and Generalization G. Wang G. Giannakis Jie Chen MLT 24 131 0 14 Aug 2018
Learning One-hidden-layer ReLU Networks via Gradient Descent Xiao Zhang Yaodong Yu Lingxiao Wang Quanquan Gu MLT 28 134 0 20 Jun 2018
Adding One Neuron Can Eliminate All Bad Local Minima Shiyu Liang Ruoyu Sun J. Lee R. Srikant 37 89 0 22 May 2018
Improved Learning of One-hidden-layer Convolutional Neural Networks with Overlaps S. Du Surbhi Goel MLT 30 17 0 20 May 2018
End-to-end Learning of a Convolutional Neural Network via Deep Tensor Decomposition Samet Oymak Mahdi Soltanolkotabi 21 12 0 16 May 2018
Understanding the Loss Surface of Neural Networks for Binary Classification Shiyu Liang Ruoyu Sun Yixuan Li R. Srikant 32 87 0 19 Feb 2018
Learning One Convolutional Layer with Overlapping Patches Surbhi Goel Adam R. Klivans Raghu Meka MLT 18 80 0 07 Feb 2018
Learning Compact Neural Networks with Regularization Samet Oymak MLT 41 39 0 05 Feb 2018
Theoretical insights into the optimization landscape of over-parameterized shallow neural networks Mahdi Soltanolkotabi Adel Javanmard J. Lee 36 415 0 16 Jul 2017
The loss surface of deep and wide neural networks Quynh N. Nguyen Matthias Hein ODL 51 283 0 26 Apr 2017
Fast and Reliable Parameter Estimation from Nonlinear Observations Samet Oymak Mahdi Soltanolkotabi 35 25 0 23 Oct 2016