Theoretical insights into the optimization landscape of over-parameterized shallow neural networks

16 July 2017

Papers citing "Theoretical insights into the optimization landscape of over-parameterized shallow neural networks"

50 / 97 papers shown

Title
Information-Theoretic Guarantees for Recovering Low-Rank Tensors from Symmetric Rank-One Measurements Eren C. Kızıldağ 58 0 0 07 Feb 2025
A Metric Topology of Deep Learning for Data Classification Jwo-Yuh Wu L. Huang Wen-Hsuan Li Chun-Hung Liu 80 0 0 20 Jan 2025
Geometry and Optimization of Shallow Polynomial Networks Yossi Arjevani Joan Bruna Joe Kileel Elzbieta Polak Matthew Trager 34 1 0 10 Jan 2025
Near-Optimal Solutions of Constrained Learning Problems Juan Elenter Luiz F. O. Chamon Alejandro Ribeiro 26 5 0 18 Mar 2024
Early Directional Convergence in Deep Homogeneous Neural Networks for Small Initializations Akshay Kumar Jarvis D. Haupt ODL 44 3 0 12 Mar 2024
Loss Landscape of Shallow ReLU-like Neural Networks: Stationary Points, Saddle Escape, and Network Embedding Zhengqing Wu Berfin Simsek Francois Ged ODL 40 0 0 08 Feb 2024
Fundamental Limits of Deep Learning-Based Binary Classifiers Trained with Hinge Loss T. Getu Georges Kaddoum M. Bennis 40 1 0 13 Sep 2023
Solving Large-scale Spatial Problems with Convolutional Neural Networks Damian Owerko Charilaos I. Kanatsoulis Alejandro Ribeiro 22 2 0 14 Jun 2023
Resilient Constrained Learning Ignacio Hounie Alejandro Ribeiro Luiz F. O. Chamon 21 9 0 04 Jun 2023
How Spurious Features Are Memorized: Precise Analysis for Random and NTK Features Simone Bombari Marco Mondelli AAML 19 4 0 20 May 2023
Provable Guarantees for Nonlinear Feature Learning in Three-Layer Neural Networks Eshaan Nichani Alexandru Damian Jason D. Lee MLT 38 13 0 11 May 2023
NTK-SAP: Improving neural network pruning by aligning training dynamics Yite Wang Dawei Li Ruoyu Sun 34 19 0 06 Apr 2023
Beyond the Universal Law of Robustness: Sharper Laws for Random Features and Neural Tangent Kernels Simone Bombari Shayan Kiyani Marco Mondelli AAML 30 10 0 03 Feb 2023
Implementation of Trained Factorization Machine Recommendation System on Quantum Annealer Chen-Yu Liu Hsin-Yu Wang Pei-Yen Liao Ching-Jui Lai Min-hsiu Hsieh LRM 21 14 0 24 Oct 2022
When Expressivity Meets Trainability: Fewer than $n$ Neurons Can Work Jiawei Zhang Yushun Zhang Mingyi Hong Ruoyu Sun Z. Luo 26 10 0 21 Oct 2022
Annihilation of Spurious Minima in Two-Layer ReLU Networks Yossi Arjevani M. Field 16 8 0 12 Oct 2022
Neural Collapse with Normalized Features: A Geometric Analysis over the Riemannian Manifold Can Yaras Peng Wang Zhihui Zhu Laura Balzano Qing Qu 25 41 0 19 Sep 2022
Intersection of Parallels as an Early Stopping Criterion Ali Vardasbi Maarten de Rijke Mostafa Dehghani MoMe 38 5 0 19 Aug 2022
Neural Networks can Learn Representations with Gradient Descent Alexandru Damian Jason D. Lee Mahdi Soltanolkotabi SSL MLT 17 114 0 30 Jun 2022
Tensor-on-Tensor Regression: Riemannian Optimization, Over-parameterization, Statistical-computational Gap, and Their Interplay Yuetian Luo Anru R. Zhang 29 19 0 17 Jun 2022
Identifying good directions to escape the NTK regime and efficiently learn low-degree plus sparse polynomials Eshaan Nichani Yunzhi Bai Jason D. Lee 27 10 0 08 Jun 2022
Non-convex online learning via algorithmic equivalence Udaya Ghai Zhou Lu Elad Hazan 14 8 0 30 May 2022
Embedding Principle in Depth for the Loss Landscape Analysis of Deep Neural Networks Zhiwei Bai Tao Luo Z. Xu Yaoyu Zhang 28 4 0 26 May 2022
On the Effective Number of Linear Regions in Shallow Univariate ReLU Networks: Convergence Guarantees and Implicit Bias Itay Safran Gal Vardi Jason D. Lee MLT 56 23 0 18 May 2022
Knowledge Distillation Meets Open-Set Semi-Supervised Learning Jing Yang Xiatian Zhu Adrian Bulat Brais Martínez Georgios Tzimiropoulos 31 7 0 13 May 2022
Convergence of gradient descent for deep neural networks S. Chatterjee ODL 21 20 0 30 Mar 2022
On the (Non-)Robustness of Two-Layer Neural Networks in Different Learning Regimes Elvis Dohmatob A. Bietti AAML 27 13 0 22 Mar 2022
A Classical-Quantum Convolutional Neural Network for Detecting Pneumonia from Chest Radiographs V. Kulkarni S. Pawale A. Kharat 19 12 0 19 Feb 2022
Random Feature Amplification: Feature Learning and Generalization in Neural Networks Spencer Frei Niladri S. Chatterji Peter L. Bartlett MLT 30 29 0 15 Feb 2022
How does unlabeled data improve generalization in self-training? A one-hidden-layer theoretical analysis Shuai Zhang M. Wang Sijia Liu Pin-Yu Chen Jinjun Xiong SSL MLT 41 22 0 21 Jan 2022
Over-Parametrized Matrix Factorization in the Presence of Spurious Stationary Points Armin Eftekhari 21 1 0 25 Dec 2021
Recur, Attend or Convolve? On Whether Temporal Modeling Matters for Cross-Domain Robustness in Action Recognition Sofia Broomé Ernest Pokropek Boyu Li Hedvig Kjellström 16 7 0 22 Dec 2021
On the Convergence of Shallow Neural Network Training with Randomly Masked Neurons Fangshuo Liao Anastasios Kyrillidis 36 16 0 05 Dec 2021
Provable Regret Bounds for Deep Online Learning and Control Xinyi Chen Edgar Minasyan Jason D. Lee Elad Hazan 36 6 0 15 Oct 2021
Why Lottery Ticket Wins? A Theoretical Perspective of Sample Complexity on Pruned Neural Networks Shuai Zhang Meng Wang Sijia Liu Pin-Yu Chen Jinjun Xiong UQCV MLT 28 13 0 12 Oct 2021
On the Global Convergence of Gradient Descent for multi-layer ResNets in the mean-field regime Zhiyan Ding Shi Chen Qin Li S. Wright MLT AI4CE 30 11 0 06 Oct 2021
Learning Transferable Parameters for Unsupervised Domain Adaptation Zhongyi Han Haoliang Sun Yilong Yin OOD 29 45 0 13 Aug 2021
A Field Guide to Federated Optimization Jianyu Wang Zachary B. Charles Zheng Xu Gauri Joshi H. B. McMahan ... Mi Zhang Tong Zhang Chunxiang Zheng Chen Zhu Wennan Zhu FedML 187 411 0 14 Jul 2021
Tensor Methods in Computer Vision and Deep Learning Yannis Panagakis Jean Kossaifi Grigorios G. Chrysos James Oldfield M. Nicolaou Anima Anandkumar S. Zafeiriou 24 119 0 07 Jul 2021
The Limitations of Large Width in Neural Networks: A Deep Gaussian Process Perspective Geoff Pleiss John P. Cunningham 28 24 0 11 Jun 2021
Understanding Overparameterization in Generative Adversarial Networks Yogesh Balaji M. Sajedi N. Kalibhat Mucong Ding Dominik Stöger Mahdi Soltanolkotabi S. Feizi AI4CE 20 21 0 12 Apr 2021
Landscape analysis for shallow neural networks: complete classification of critical points for affine target functions Patrick Cheridito Arnulf Jentzen Florian Rossmannek 24 10 0 19 Mar 2021
Recent Advances on Neural Network Pruning at Initialization Huan Wang Can Qin Yue Bai Yulun Zhang Yun Fu CVBM 33 64 0 11 Mar 2021
Fundamental Tradeoffs in Distributionally Adversarial Training M. Mehrabi Adel Javanmard Ryan A. Rossi Anup B. Rao Tung Mai AAML 20 17 0 15 Jan 2021
Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning Zeyuan Allen-Zhu Yuanzhi Li FedML 38 355 0 17 Dec 2020
Align, then memorise: the dynamics of learning with feedback alignment Maria Refinetti Stéphane dÁscoli Ruben Ohana Sebastian Goldt 26 36 0 24 Nov 2020
Are wider nets better given the same number of parameters? A. Golubeva Behnam Neyshabur Guy Gur-Ari 21 44 0 27 Oct 2020
Computational Separation Between Convolutional and Fully-Connected Networks Eran Malach Shai Shalev-Shwartz 19 26 0 03 Oct 2020
Understanding Self-supervised Learning with Dual Deep Networks Yuandong Tian Lantao Yu Xinlei Chen Surya Ganguli SSL 13 78 0 01 Oct 2020
A Survey on Large-scale Machine Learning Meng Wang Weijie Fu Xiangnan He Shijie Hao Xindong Wu 14 109 0 10 Aug 2020