Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Loss

11 February 2020

Papers citing "Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Loss"

50 / 252 papers shown

Title
Implicit Regularization in Tensor Factorization Noam Razin Asaf Maman Nadav Cohen 33 48 0 19 Feb 2021
On the Implicit Bias of Initialization Shape: Beyond Infinitesimal Mirror Descent Shahar Azulay E. Moroshko Mor Shpigel Nacson Blake E. Woodworth Nathan Srebro Amir Globerson Daniel Soudry AI4CE 33 73 0 19 Feb 2021
Bridging the Gap Between Adversarial Robustness and Optimization Bias Fartash Faghri Sven Gowal C. N. Vasconcelos David J. Fleet Fabian Pedregosa Nicolas Le Roux AAML 195 7 0 17 Feb 2021
Painless step size adaptation for SGD I. Kulikovskikh Tarzan Legović 25 0 0 01 Feb 2021
On the Proof of Global Convergence of Gradient Descent for Deep ReLU Networks with Linear Widths Quynh N. Nguyen 41 49 0 24 Jan 2021
Towards Understanding Learning in Neural Networks with Linear Teachers Roei Sarussi Alon Brutzkus Amir Globerson FedML MLT 55 20 0 07 Jan 2021
Particle Dual Averaging: Optimization of Mean Field Neural Networks with Global Convergence Rate Analysis Atsushi Nitanda Denny Wu Taiji Suzuki 8 29 0 31 Dec 2020
Perspective: A Phase Diagram for Deep Learning unifying Jamming, Feature Learning and Lazy Training Mario Geiger Leonardo Petrini M. Wyart DRL 28 11 0 30 Dec 2020
Mathematical Models of Overparameterized Neural Networks Cong Fang Hanze Dong Tong Zhang 27 22 0 27 Dec 2020
Towards Resolving the Implicit Bias of Gradient Descent for Matrix Factorization: Greedy Low-Rank Learning Zhiyuan Li Yuping Luo Kaifeng Lyu 20 120 0 17 Dec 2020
On the emergence of simplex symmetry in the final and penultimate layers of neural network classifiers E. Weinan Stephan Wojtowytsch 28 42 0 10 Dec 2020
Implicit Regularization in ReLU Networks with the Square Loss Gal Vardi Ohad Shamir 14 48 0 09 Dec 2020
Benefit of deep learning with non-convex noisy gradient descent: Provable excess risk bound and superiority to kernel methods Taiji Suzuki Shunta Akiyama MLT 21 12 0 06 Dec 2020
Feature Learning in Infinite-Width Neural Networks Greg Yang J. E. Hu MLT 11 147 0 30 Nov 2020
Implicit bias of deep linear networks in the large learning rate phase Wei Huang Weitao Du R. Xu Chunrui Liu 24 2 0 25 Nov 2020
Balance Regularized Neural Network Models for Causal Effect Estimation Mehrdad Farajtabar Andrew Lee Yuanjian Feng Vishal Gupta Peter Dolan Harish Chandran M. Szummer CML 6 6 0 23 Nov 2020
Implicit bias of any algorithm: bounding bias via margin Elvis Dohmatob 9 0 0 12 Nov 2020
Direction Matters: On the Implicit Bias of Stochastic Gradient Descent with Moderate Learning Rate Jingfeng Wu Difan Zou Vladimir Braverman Quanquan Gu 8 18 0 04 Nov 2020
Inductive Bias of Gradient Descent for Weight Normalized Smooth Homogeneous Neural Nets Depen Morwani H. G. Ramaswamy 9 3 0 24 Oct 2020
Global optimality of softmax policy gradient with single hidden layer neural networks in the mean-field regime Andrea Agazzi Jianfeng Lu 13 15 0 22 Oct 2020
The Deep Bootstrap Framework: Good Online Learners are Good Offline Generalizers Preetum Nakkiran Behnam Neyshabur Hanie Sedghi OffRL 29 11 0 16 Oct 2020
Temperature check: theory and practice for training models with softmax-cross-entropy losses Atish Agarwala Jeffrey Pennington Yann N. Dauphin S. Schoenholz UQCV 16 32 0 14 Oct 2020
A Unifying View on Implicit Bias in Training Linear Neural Networks Chulhee Yun Shankar Krishnan H. Mobahi MLT 13 80 0 06 Oct 2020
A priori estimates for classification problems using neural networks E. Weinan Stephan Wojtowytsch 6 8 0 28 Sep 2020
Towards a Mathematical Understanding of Neural Network-Based Machine Learning: what we know and what we don't E. Weinan Chao Ma Stephan Wojtowytsch Lei Wu AI4CE 22 133 0 22 Sep 2020
Distributional Generalization: A New Kind of Generalization Preetum Nakkiran Yamini Bansal OOD 23 41 0 17 Sep 2020
Deep Networks and the Multiple Manifold Problem Sam Buchanan D. Gilboa John N. Wright 166 39 0 25 Aug 2020
A Dynamical Central Limit Theorem for Shallow Neural Networks Zhengdao Chen Grant M. Rotskoff Joan Bruna Eric Vanden-Eijnden 18 29 0 21 Aug 2020
A Functional Perspective on Learning Symmetric Functions with Neural Networks Aaron Zweig Joan Bruna 11 21 0 16 Aug 2020
Large-time asymptotics in deep learning Carlos Esteve Borjan Geshkovski Dario Pighin Enrique Zuazua 14 34 0 06 Aug 2020
On the Banach spaces associated with multi-layer ReLU networks: Function representation, approximation theory and gradient descent dynamics E. Weinan Stephan Wojtowytsch MLT 13 53 0 30 Jul 2020
Geometric compression of invariant manifolds in neural nets J. Paccolat Leonardo Petrini Mario Geiger Kevin Tyloo M. Wyart MLT 55 34 0 22 Jul 2020
Understanding Implicit Regularization in Over-Parameterized Single Index Model Jianqing Fan Zhuoran Yang Mengxin Yu 24 16 0 16 Jul 2020
Global Convergence of Second-order Dynamics in Two-layer Neural Networks Walid Krichene Kenneth F. Caluya A. Halder MLT 8 5 0 14 Jul 2020
Implicit Bias in Deep Linear Classification: Initialization Scale vs Training Accuracy E. Moroshko Suriya Gunasekar Blake E. Woodworth J. Lee Nathan Srebro Daniel Soudry 35 85 0 13 Jul 2020
The Surprising Simplicity of the Early-Time Learning Dynamics of Neural Networks Wei Hu Lechao Xiao Ben Adlam Jeffrey Pennington 23 62 0 25 Jun 2020
The Quenching-Activation Behavior of the Gradient Descent Dynamics for Two-layer Neural Network Models Chao Ma Lei Wu E. Weinan MLT 23 10 0 25 Jun 2020
Implicitly Maximizing Margins with the Hinge Loss Justin Lizama 13 1 0 25 Jun 2020
When Do Neural Networks Outperform Kernel Methods? Behrooz Ghorbani Song Mei Theodor Misiakiewicz Andrea Montanari 23 184 0 24 Jun 2020
Spectral Bias and Task-Model Alignment Explain Generalization in Kernel Regression and Infinitely Wide Neural Networks Abdulkadir Canatar Blake Bordelon Cengiz Pehlevan 19 181 0 23 Jun 2020
When Does Preconditioning Help or Hurt Generalization? S. Amari Jimmy Ba Roger C. Grosse Xuechen Li Atsushi Nitanda Taiji Suzuki Denny Wu Ji Xu 36 32 0 18 Jun 2020
On Sparsity in Overparametrised Shallow ReLU Networks Jaume de Dios Joan Bruna 19 14 0 18 Jun 2020
The Pitfalls of Simplicity Bias in Neural Networks Harshay Shah Kaustav Tamuly Aditi Raghunathan Prateek Jain Praneeth Netrapalli AAML 18 348 0 13 Jun 2020
Directional convergence and alignment in deep learning Ziwei Ji Matus Telgarsky 14 162 0 11 Jun 2020
Structure preserving deep learning E. Celledoni Matthias Joachim Ehrhardt Christian Etmann R. McLachlan B. Owren Carola-Bibiane Schönlieb Ferdia Sherry AI4CE 15 44 0 05 Jun 2020
On the Convergence of Gradient Descent Training for Two-layer ReLU-networks in the Mean Field Regime Stephan Wojtowytsch MLT 24 49 0 27 May 2020
Can Shallow Neural Networks Beat the Curse of Dimensionality? A mean field training perspective Stephan Wojtowytsch E. Weinan MLT 26 48 0 21 May 2020
Implicit Regularization in Deep Learning May Not Be Explainable by Norms Noam Razin Nadav Cohen 24 155 0 13 May 2020
Estimating g-Leakage via Machine Learning Marco Romanelli K. Chatzikokolakis C. Palamidessi Pablo Piantanida MIACV 12 12 0 09 May 2020
Random Features for Kernel Approximation: A Survey on Algorithms, Theory, and Beyond Fanghui Liu Xiaolin Huang Yudong Chen Johan A. K. Suykens BDL 44 172 0 23 Apr 2020