Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Loss

11 February 2020

Papers citing "Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Loss"

50 / 252 papers shown

Title
Sign-In to the Lottery: Reparameterizing Sparse Training From Scratch Advait Gadhikar Tom Jacobs Chao Zhou R. Burkholz 32 0 0 17 Apr 2025
Gradient Descent Robustly Learns the Intrinsic Dimension of Data in Training Convolutional Neural Networks Chenyang Zhang Peifeng Gao Difan Zou Yuan Cao OOD MLT 59 0 0 11 Apr 2025
When Do Transformers Outperform Feedforward and Recurrent Networks? A Statistical Perspective Alireza Mousavi-Hosseini Clayton Sanford Denny Wu Murat A. Erdogdu 48 0 0 14 Mar 2025
Learning richness modulates equality reasoning in neural networks William L. Tong C. Pehlevan 47 0 0 12 Mar 2025
Curse of Dimensionality in Neural Network Optimization Sanghoon Na Haizhao Yang 56 0 0 07 Feb 2025
Optimization Insights into Deep Diagonal Linear Networks Hippolyte Labarrière C. Molinari Lorenzo Rosasco S. Villa Cristian Vega 76 0 0 21 Dec 2024
Random Feature Models with Learnable Activation Functions Zailin Ma Jiansheng Yang Yaodong Yang 77 0 0 29 Nov 2024
Robust Feature Learning for Multi-Index Models in High Dimensions Alireza Mousavi-Hosseini Adel Javanmard Murat A. Erdogdu OOD AAML 44 1 0 21 Oct 2024
SimBa: Simplicity Bias for Scaling Up Parameters in Deep Reinforcement Learning Hojoon Lee Dongyoon Hwang Donghu Kim Hyunseung Kim Jun Jet Tai K. Subramanian Peter R. Wurman Jaegul Choo Peter Stone Takuma Seno OffRL 72 6 0 13 Oct 2024
Convex Distillation: Efficient Compression of Deep Networks via Convex Optimization Prateek Varshney Mert Pilanci 40 0 0 09 Oct 2024
Simplicity bias and optimization threshold in two-layer ReLU networks Etienne Boursier Nicolas Flammarion 31 2 0 03 Oct 2024
Solving High-Dimensional Partial Integral Differential Equations: The Finite Expression Method Gareth Hardwick Senwei Liang Haizhao Yang 36 1 0 01 Oct 2024
From Lazy to Rich: Exact Learning Dynamics in Deep Linear Networks Clémentine Dominé Nicolas Anguita A. Proca Lukas Braun D. Kunin P. Mediano Andrew M. Saxe 38 3 0 22 Sep 2024
The Optimality of (Accelerated) SGD for High-Dimensional Quadratic Optimization Haihan Zhang Yuanshi Liu Qianwen Chen Cong Fang 38 0 0 15 Sep 2024
Mask in the Mirror: Implicit Sparsification Tom Jacobs R. Burkholz 47 3 0 19 Aug 2024
Learning Multi-Index Models with Neural Networks via Mean-Field Langevin Dynamics Alireza Mousavi-Hosseini Denny Wu Murat A. Erdogdu MLT AI4CE 35 6 0 14 Aug 2024
How DNNs break the Curse of Dimensionality: Compositionality and Symmetry Learning Arthur Jacot Seok Hoan Choi Yuxiao Wen AI4CE 91 2 0 08 Jul 2024
Implicit Bias of Mirror Flow on Separable Data Scott Pesme Radu-Alexandru Dragomir Nicolas Flammarion 39 1 0 18 Jun 2024
The Implicit Bias of Adam on Separable Data Chenyang Zhang Difan Zou Yuan Cao AI4CE 45 7 0 15 Jun 2024
Large Stepsize Gradient Descent for Non-Homogeneous Two-Layer Networks: Margin Improvement and Fast Optimization Yuhang Cai Jingfeng Wu Song Mei Michael Lindsey Peter L. Bartlett 34 2 0 12 Jun 2024
Stable Minima Cannot Overfit in Univariate ReLU Networks: Generalization by Large Step Sizes Dan Qiao Kaiqi Zhang Esha Singh Daniel Soudry Yu-Xiang Wang NoLa 36 3 0 10 Jun 2024
Get rich quick: exact solutions reveal how unbalanced initializations promote rapid feature learning D. Kunin Allan Raventós Clémentine Dominé Feng Chen David Klindt Andrew M. Saxe Surya Ganguli MLT 48 15 0 10 Jun 2024
ReLUs Are Sufficient for Learning Implicit Neural Representations Joseph Shenouda Yamin Zhou Robert D. Nowak 30 5 0 04 Jun 2024
Improving Generalization and Convergence by Enhancing Implicit Regularization Mingze Wang Haotian He Jinbo Wang Zilin Wang Guanhua Huang Feiyu Xiong Zhiyu Li E. Weinan Lei Wu 45 6 0 31 May 2024
Mixed Dynamics In Linear Networks: Unifying the Lazy and Active Regimes Zhenfeng Tu Santiago Aranguri Arthur Jacot 31 8 0 27 May 2024
Hamiltonian Mechanics of Feature Learning: Bottleneck Structure in Leaky ResNets Arthur Jacot Alexandre Kaiser 36 0 0 27 May 2024
When does compositional structure yield compositional generalization? A kernel theory Samuel Lippl Kim Stachenfeld NAI CoGe 73 5 0 26 May 2024
Can Implicit Bias Imply Adversarial Robustness? Hancheng Min René Vidal 39 3 0 24 May 2024
Keep the Momentum: Conservation Laws beyond Euclidean Gradient Flows Sibylle Marcotte Rémi Gribonval Gabriel Peyré 26 0 0 21 May 2024
Regularized Gauss-Newton for Optimizing Overparameterized Neural Networks Adeyemi Damilare Adeoye Philipp Christian Petersen Alberto Bemporad 28 1 0 23 Apr 2024
Matching the Statistical Query Lower Bound for k-sparse Parity Problems with Stochastic Gradient Descent Yiwen Kou Zixiang Chen Quanquan Gu Sham Kakade 44 0 0 18 Apr 2024
Implementation and Evaluation of a Gradient Descent-Trained Defensible Blackboard Architecture System Jordan Milbrath Jonathan Rivard Jeremy Straub 17 1 0 17 Apr 2024
Posterior Uncertainty Quantification in Neural Networks using Data Augmentation Luhuan Wu Sinead Williamson UQCV 40 6 0 18 Mar 2024
Transfer Learning Beyond Bounded Density Ratios Alkis Kalavasis Ilias Zadik Manolis Zampetakis 50 4 0 18 Mar 2024
Generalization of Scaled Deep ResNets in the Mean-Field Regime Yihang Chen Fanghui Liu Yiping Lu Grigorios G. Chrysos V. Cevher 41 2 0 14 Mar 2024
Early Directional Convergence in Deep Homogeneous Neural Networks for Small Initializations Akshay Kumar Jarvis Haupt ODL 44 3 0 12 Mar 2024
When Representations Align: Universality in Representation Learning Dynamics Loek van Rossem Andrew M. Saxe AI4CE 48 4 0 14 Feb 2024
Depth Separation in Norm-Bounded Infinite-Width Neural Networks Suzanna Parkinson Greg Ongie Rebecca Willett Ohad Shamir Nathan Srebro MDE 50 2 0 13 Feb 2024
Implicit Bias in Noisy-SGD: With Applications to Differentially Private Training Tom Sander Maxime Sylvestre Alain Durmus 31 1 0 13 Feb 2024
How Uniform Random Weights Induce Non-uniform Bias: Typical Interpolating Neural Networks Generalize with Narrow Teachers G. Buzaglo I. Harel Mor Shpigel Nacson Alon Brutzkus Nathan Srebro Daniel Soudry 62 3 0 09 Feb 2024
Implicit Bias and Fast Convergence Rates for Self-attention Bhavya Vasudeva Puneesh Deora Christos Thrampoulidis 34 13 0 08 Feb 2024
Task structure and nonlinearity jointly determine learned representational geometry Matteo Alleman Jack W. Lindsey Stefano Fusi 38 8 0 24 Jan 2024
Learning from higher-order statistics, efficiently: hypothesis tests, random features, and neural networks Eszter Székely Lorenzo Bardone Federica Gerace Sebastian Goldt 34 2 0 22 Dec 2023
The Convex Landscape of Neural Networks: Characterizing Global Optima and Stationary Points via Lasso Models Tolga Ergen Mert Pilanci 16 2 0 19 Dec 2023
Space-Time Approximation with Shallow Neural Networks in Fourier Lebesgue spaces Ahmed Abdeljawad Thomas Dittrich 31 2 0 13 Dec 2023
Dichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokking Kaifeng Lyu Jikai Jin Zhiyuan Li Simon S. Du Jason D. Lee Wei Hu AI4CE 44 32 0 30 Nov 2023
GGNNs : Generalizing GNNs using Residual Connections and Weighted Message Passing Abhinav Raghuvanshi K. Malleshappa AI4CE GNN 29 0 0 26 Nov 2023
Applying statistical learning theory to deep learning Cédric Gerbelot Avetik G. Karagulyan Stefani Karp Kavya Ravichandran Menachem Stern Nathan Srebro FedML 16 2 0 26 Nov 2023
Achieving Margin Maximization Exponentially Fast via Progressive Norm Rescaling Mingze Wang Zeping Min Lei Wu 33 3 0 24 Nov 2023
Feature emergence via margin maximization: case studies in algebraic tasks Depen Morwani Benjamin L. Edelman Costin-Andrei Oncescu Rosie Zhao Sham Kakade 39 14 0 13 Nov 2023