Implicit Sparse Regularization: The Impact of Depth and Early Stopping

v1v2 (latest)

Implicit Sparse Regularization: The Impact of Depth and Early Stopping

12 August 2021

Thanh V. Nguyen

ArXiv (abs)PDF HTML

Papers citing "Implicit Sparse Regularization: The Impact of Depth and Early Stopping"

14 / 14 papers shown

Title
Deep Weight Factorization: Sparse Learning Through the Lens of Artificial Symmetries Chris Kolb T. Weber Bernd Bischl David Rügamer 326 1 0 04 Feb 2025
Mask in the Mirror: Implicit Sparsification Tom Jacobs R. Burkholz 173 4 0 19 Aug 2024
Towards Resolving the Implicit Bias of Gradient Descent for Matrix Factorization: Greedy Low-Rank Learning Zhiyuan Li Yuping Luo Kaifeng Lyu 92 130 0 17 Dec 2020
The Implicit Bias of Depth: How Incremental Learning Drives Generalization Daniel Gissin Shai Shalev-Shwartz Amit Daniely AI4CE 90 81 0 26 Sep 2019
Implicit Regularization for Optimal Sparse Recovery Tomas Vaskevicius Varun Kanade Patrick Rebeschini 49 103 0 11 Sep 2019
Gradient Dynamics of Shallow Univariate ReLU Networks Francis Williams Matthew Trager Claudio Silva Daniele Panozzo Denis Zorin Joan Bruna 62 80 0 18 Jun 2019
Kernel and Rich Regimes in Overparametrized Models Blake E. Woodworth Suriya Gunasekar Pedro H. P. Savarese E. Moroshko Itay Golan Jason D. Lee Daniel Soudry Nathan Srebro 80 364 0 13 Jun 2019
Implicit Regularization in Deep Matrix Factorization Sanjeev Arora Nadav Cohen Wei Hu Yuping Luo AI4CE 85 509 0 31 May 2019
Reconciling modern machine learning practice and the bias-variance trade-off M. Belkin Daniel J. Hsu Siyuan Ma Soumik Mandal 242 1,655 0 28 Dec 2018
Implicit Bias of Gradient Descent on Linear Convolutional Networks Suriya Gunasekar Jason D. Lee Daniel Soudry Nathan Srebro MDE 124 413 0 01 Jun 2018
Iterate averaging as regularization for stochastic gradient descent Gergely Neu Lorenzo Rosasco MoMe 76 61 0 22 Feb 2018
Geometry of Optimization and Implicit Regularization in Deep Learning Behnam Neyshabur Ryota Tomioka Ruslan Salakhutdinov Nathan Srebro AI4CE 67 133 0 08 May 2017
Understanding deep learning requires rethinking generalization Chiyuan Zhang Samy Bengio Moritz Hardt Benjamin Recht Oriol Vinyals HAI 345 4,636 0 10 Nov 2016
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks Andrew M. Saxe James L. McClelland Surya Ganguli ODL 181 1,849 0 20 Dec 2013