Lecture Notes on Linear Neural Networks: A Tale of Optimization and Generalization in Deep Learning

25 August 2024

Papers citing "Lecture Notes on Linear Neural Networks: A Tale of Optimization and Generalization in Deep Learning"

26 / 26 papers shown

Title
Understanding Incremental Learning of Gradient Descent: A Fine-grained Analysis of Matrix Sensing Jikai Jin Zhiyuan Li Kaifeng Lyu S. Du Jason D. Lee MLT 93 37 0 27 Jan 2023
On the Ability of Graph Neural Networks to Model Interactions Between Vertices Noam Razin Tom Verbin Nadav Cohen 58 11 0 29 Nov 2022
Convergence of gradient descent for learning linear neural networks Gabin Maxime Nguegnang Holger Rauhut Ulrich Terstiege MLT 47 18 0 04 Aug 2021
On the Implicit Bias of Initialization Shape: Beyond Infinitesimal Mirror Descent Shahar Azulay E. Moroshko Mor Shpigel Nacson Blake E. Woodworth Nathan Srebro Amir Globerson Daniel Soudry AI4CE 77 74 0 19 Feb 2021
Towards Resolving the Implicit Bias of Gradient Descent for Matrix Factorization: Greedy Low-Rank Learning Zhiyuan Li Yuping Luo Kaifeng Lyu 92 130 0 17 Dec 2020
Implicit Rank-Minimizing Autoencoder Li Jing Jure Zbontar Yann LeCun SSL DRL 70 48 0 01 Oct 2020
DO-Conv: Depthwise Over-parameterized Convolutional Layer Jinming Cao Yangyan Li Mingchao Sun Ying-Cong Chen Dani Lischinski Daniel Cohen-Or Baoquan Chen Changhe Tu OOD 66 172 0 22 Jun 2020
Deep Polynomial Neural Networks Grigorios G. Chrysos Stylianos Moschoglou Giorgos Bouritsas Jiankang Deng Yannis Panagakis Stefanos Zafeiriou 63 93 0 20 Jun 2020
Learning deep linear neural networks: Riemannian gradient flows and convergence to global minimizers B. Bah Holger Rauhut Ulrich Terstiege Michael Westdickenberg MLT 37 66 0 12 Oct 2019
Blind Super-Resolution Kernel Estimation using an Internal-GAN Sefi Bell-Kligler Assaf Shocher Michal Irani SupR 64 455 0 14 Sep 2019
Implicit Regularization in Deep Matrix Factorization Sanjeev Arora Nadav Cohen Wei Hu Yuping Luo AI4CE 87 509 0 31 May 2019
Generalized Tensor Models for Recurrent Neural Networks Valentin Khrulkov Oleksii Hrinchuk Ivan Oseledets GNN 36 25 0 30 Jan 2019
Width Provably Matters in Optimization for Deep Linear Neural Networks S. Du Wei Hu 72 95 0 24 Jan 2019
An analytic theory of generalization dynamics and transfer learning in deep linear networks Andrew Kyle Lampinen Surya Ganguli OOD 82 131 0 27 Sep 2018
Quantum Entanglement in Deep Learning Architectures Yoav Levine Or Sharir Nadav Cohen Amnon Shashua 79 182 0 26 Mar 2018
On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization Sanjeev Arora Nadav Cohen Elad Hazan 105 488 0 19 Feb 2018
On the Long-Term Memory of Deep Recurrent Networks Yoav Levine Or Sharir Alon Ziv Amnon Shashua 49 24 0 25 Oct 2017
Analysis and Design of Convolutional Networks via Hierarchical Tensor Decompositions Nadav Cohen Or Sharir Yoav Levine Ronen Tamari David Yakira Amnon Shashua 100 38 0 05 May 2017
Deep Learning and Quantum Entanglement: Fundamental Connections with Implications to Network Design Yoav Levine David Yakira Nadav Cohen Amnon Shashua 104 126 0 05 Apr 2017
Identity Matters in Deep Learning Moritz Hardt Tengyu Ma OOD 89 399 0 14 Nov 2016
Understanding deep learning requires rethinking generalization Chiyuan Zhang Samy Bengio Moritz Hardt Benjamin Recht Oriol Vinyals HAI 345 4,636 0 10 Nov 2016
Matrix Completion has No Spurious Local Minimum Rong Ge Jason D. Lee Tengyu Ma 114 599 0 24 May 2016
Deep SimNets Nadav Cohen Or Sharir Amnon Shashua 69 46 0 09 Jun 2015
Escaping From Saddle Points --- Online Stochastic Gradient for Tensor Decomposition Rong Ge Furong Huang Chi Jin Yang Yuan 140 1,059 0 06 Mar 2015
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks Andrew M. Saxe James L. McClelland Surya Ganguli ODL 183 1,852 0 20 Dec 2013
Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization Benjamin Recht Maryam Fazel P. Parrilo 419 3,771 0 28 Jun 2007