v1v2v3 (latest)

A Unifying View on Implicit Bias in Training Linear Neural Networks

6 October 2020

Papers citing "A Unifying View on Implicit Bias in Training Linear Neural Networks"

33 / 33 papers shown

Title
The late-stage training dynamics of (stochastic) subgradient descent on homogeneous neural networks Sholom Schechtman Nicolas Schreuder 441 0 0 08 Feb 2025
Optimization Insights into Deep Diagonal Linear Networks Hippolyte Labarrière C. Molinari Lorenzo Rosasco S. Villa Cristian Vega 173 1 0 21 Dec 2024
Inductive Bias of Multi-Channel Linear Convolutional Networks with Bounded Weight Norm Meena Jagadeesan Ilya P. Razenshteyn Suriya Gunasekar 49 21 0 24 Feb 2021
Implicit Bias in Deep Linear Classification: Initialization Scale vs Training Accuracy E. Moroshko Suriya Gunasekar Blake E. Woodworth Jason D. Lee Nathan Srebro Daniel Soudry 71 86 0 13 Jul 2020
Directional convergence and alignment in deep learning Ziwei Ji Matus Telgarsky 59 171 0 11 Jun 2020
Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Loss Lénaïc Chizat Francis R. Bach MLT 127 341 0 11 Feb 2020
Provable Benefit of Orthogonal Initialization in Optimizing Deep Linear Networks Wei Hu Lechao Xiao Jeffrey Pennington 64 115 0 16 Jan 2020
Global Convergence of Gradient Descent for Deep Linear Residual Networks Lei Wu Qingcan Wang Chao Ma ODL AI4CE 74 22 0 02 Nov 2019
The Implicit Bias of Depth: How Incremental Learning Drives Generalization Daniel Gissin Shai Shalev-Shwartz Amit Daniely AI4CE 93 81 0 26 Sep 2019
Gradient Descent Maximizes the Margin of Homogeneous Neural Networks Kaifeng Lyu Jian Li 89 336 0 13 Jun 2019
Kernel and Rich Regimes in Overparametrized Models Blake E. Woodworth Suriya Gunasekar Pedro H. P. Savarese E. Moroshko Itay Golan Jason D. Lee Daniel Soudry Nathan Srebro 80 366 0 13 Jun 2019
Characterizing the implicit bias via a primal-dual analysis Ziwei Ji Matus Telgarsky 53 20 0 11 Jun 2019
Implicit Regularization in Deep Matrix Factorization Sanjeev Arora Nadav Cohen Wei Hu Yuping Luo AI4CE 87 509 0 31 May 2019
Lexicographic and Depth-Sensitive Margins in Homogeneous and Non-Homogeneous Deep Models Mor Shpigel Nacson Suriya Gunasekar Jason D. Lee Nathan Srebro Daniel Soudry 81 94 0 17 May 2019
Identity Crisis: Memorization and Generalization under Extreme Overparameterization Chiyuan Zhang Samy Bengio Moritz Hardt Michael C. Mozer Y. Singer 51 90 0 13 Feb 2019
Towards moderate overparameterization: global convergence guarantees for training shallow neural networks Samet Oymak Mahdi Soltanolkotabi 52 322 0 12 Feb 2019
Width Provably Matters in Optimization for Deep Linear Neural Networks S. Du Wei Hu 72 95 0 24 Jan 2019
On Lazy Training in Differentiable Programming Lénaïc Chizat Edouard Oyallon Francis R. Bach 111 839 0 19 Dec 2018
Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks Difan Zou Yuan Cao Dongruo Zhou Quanquan Gu ODL 193 448 0 21 Nov 2018
A Convergence Theory for Deep Learning via Over-Parameterization Zeyuan Allen-Zhu Yuanzhi Li Zhao Song AI4CE ODL 266 1,469 0 09 Nov 2018
Gradient Descent Finds Global Minima of Deep Neural Networks S. Du Jason D. Lee Haochuan Li Liwei Wang Masayoshi Tomizuka ODL 216 1,135 0 09 Nov 2018
A Convergence Analysis of Gradient Descent for Deep Linear Neural Networks Sanjeev Arora Nadav Cohen Noah Golowich Wei Hu 125 293 0 04 Oct 2018
Gradient Descent Provably Optimizes Over-parameterized Neural Networks S. Du Xiyu Zhai Barnabás Póczós Aarti Singh MLT ODL 227 1,276 0 04 Oct 2018
Gradient descent aligns the layers of deep linear networks Ziwei Ji Matus Telgarsky 123 257 0 04 Oct 2018
Neural Tangent Kernel: Convergence and Generalization in Neural Networks Arthur Jacot Franck Gabriel Clément Hongler 273 3,219 0 20 Jun 2018
Stochastic Gradient Descent on Separable Data: Exact Convergence with a Fixed Learning Rate Mor Shpigel Nacson Nathan Srebro Daniel Soudry FedML MLT 80 102 0 05 Jun 2018
Implicit Bias of Gradient Descent on Linear Convolutional Networks Suriya Gunasekar Jason D. Lee Daniel Soudry Nathan Srebro MDE 124 413 0 01 Jun 2018
Convergence of Gradient Descent on Separable Data Mor Shpigel Nacson Jason D. Lee Suriya Gunasekar Pedro H. P. Savarese Nathan Srebro Daniel Soudry 76 169 0 05 Mar 2018
Characterizing Implicit Bias in Terms of Optimization Geometry Suriya Gunasekar Jason D. Lee Daniel Soudry Nathan Srebro AI4CE 73 411 0 22 Feb 2018
On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization Sanjeev Arora Nadav Cohen Elad Hazan 105 488 0 19 Feb 2018
Gradient descent with identity initialization efficiently learns positive definite linear transformations by deep residual networks Peter L. Bartlett D. Helmbold Philip M. Long 94 117 0 16 Feb 2018
The Implicit Bias of Gradient Descent on Separable Data Daniel Soudry Elad Hoffer Mor Shpigel Nacson Suriya Gunasekar Nathan Srebro 161 923 0 27 Oct 2017
Implicit Regularization in Matrix Factorization Suriya Gunasekar Blake E. Woodworth Srinadh Bhojanapalli Behnam Neyshabur Nathan Srebro 79 493 0 25 May 2017