Implicit Bias in Deep Linear Classification: Initialization Scale vs
Training Accuracy

Implicit Bias in Deep Linear Classification: Initialization Scale vs Training Accuracy

13 July 2020

Suriya Gunasekar

Blake E. Woodworth

Papers citing "Implicit Bias in Deep Linear Classification: Initialization Scale vs Training Accuracy"

19 / 19 papers shown

Title
Optimization Insights into Deep Diagonal Linear Networks Hippolyte Labarrière C. Molinari Lorenzo Rosasco S. Villa Cristian Vega 76 0 0 21 Dec 2024
Compressible Dynamics in Deep Overparameterized Low-Rank Learning & Adaptation Can Yaras Peng Wang Laura Balzano Qing Qu AI4CE 37 12 0 06 Jun 2024
Deconstructing Data Reconstruction: Multiclass, Weight Decay and General Losses G. Buzaglo Niv Haim Gilad Yehudai Gal Vardi Yakir Oz Yaniv Nikankin Michal Irani 31 10 0 04 Jul 2023
Generalization on the Unseen, Logic Reasoning and Degree Curriculum Emmanuel Abbe Samy Bengio Aryo Lotfi Kevin Rizk LRM 39 48 0 30 Jan 2023
SGD with Large Step Sizes Learns Sparse Features Maksym Andriushchenko Aditya Varre Loucas Pillaud-Vivien Nicolas Flammarion 45 56 0 11 Oct 2022
Implicit Bias of Large Depth Networks: a Notion of Rank for Nonlinear Functions Arthur Jacot 36 25 0 29 Sep 2022
Magnitude and Angle Dynamics in Training Single ReLU Neurons Sangmin Lee Byeongsu Sim Jong Chul Ye MLT 96 6 0 27 Sep 2022
On the Implicit Bias in Deep-Learning Algorithms Gal Vardi FedML AI4CE 34 72 0 26 Aug 2022
Implicit Bias of Gradient Descent on Reparametrized Models: On Equivalence to Mirror Descent Zhiyuan Li Tianhao Wang Jason D. Lee Sanjeev Arora 37 27 0 08 Jul 2022
Reconstructing Training Data from Trained Neural Networks Niv Haim Gal Vardi Gilad Yehudai Ohad Shamir Michal Irani 40 132 0 15 Jun 2022
Thinking Outside the Ball: Optimal Learning with Gradient Descent for Generalized Linear Stochastic Convex Optimization I Zaghloul Amir Roi Livni Nathan Srebro 24 6 0 27 Feb 2022
Implicit Regularization in Hierarchical Tensor Factorization and Deep Convolutional Neural Networks Noam Razin Asaf Maman Nadav Cohen 46 29 0 27 Jan 2022
Parallel Deep Neural Networks Have Zero Duality Gap Yifei Wang Tolga Ergen Mert Pilanci 79 10 0 13 Oct 2021
AIR-Net: Adaptive and Implicit Regularization Neural Network for Matrix Completion Zhemin Li Tao Sun Hongxia Wang Bao Wang 42 6 0 12 Oct 2021
On Margin Maximization in Linear and ReLU Networks Gal Vardi Ohad Shamir Nathan Srebro 50 28 0 06 Oct 2021
A Theoretical Analysis of Fine-tuning with Linear Teachers Gal Shachaf Alon Brutzkus Amir Globerson 31 17 0 04 Jul 2021
Experiments with Rich Regime Training for Deep Learning Xinyan Li A. Banerjee 29 2 0 26 Feb 2021
On the Implicit Bias of Initialization Shape: Beyond Infinitesimal Mirror Descent Shahar Azulay E. Moroshko Mor Shpigel Nacson Blake E. Woodworth Nathan Srebro Amir Globerson Daniel Soudry AI4CE 30 73 0 19 Feb 2021
When Will Gradient Methods Converge to Max-margin Classifier under ReLU Models? Tengyu Xu Yi Zhou Kaiyi Ji Yingbin Liang 23 19 0 12 Jun 2018