Benign Overfitting and Grokking in ReLU Networks for XOR Cluster Data

Benign Overfitting and Grokking in ReLU Networks for XOR Cluster Data

4 October 2023

Gal Vardi

ArXiv (abs)PDF HTML

Papers citing "Benign Overfitting and Grokking in ReLU Networks for XOR Cluster Data"

14 / 14 papers shown

Title
When does compositional structure yield compositional generalization? A kernel theory Samuel Lippl Kim Stachenfeld NAI CoGe 230 10 0 26 May 2024
From Tempered to Benign Overfitting in ReLU Neural Networks Guy Kornowski Gilad Yehudai Ohad Shamir 77 13 0 24 May 2023
A Tale of Two Circuits: Grokking as Competition of Sparse and Dense Subnetworks William Merrill Nikolaos Tsilivis Aman Shukla 67 54 0 21 Mar 2023
Unifying Grokking and Double Descent Peter W. Battaglia David Raposo Kelsey 83 32 0 10 Mar 2023
Benign Overfitting in Linear Classifiers and Leaky ReLU Networks from KKT Conditions for Margin Maximization Spencer Frei Gal Vardi Peter L. Bartlett Nathan Srebro 78 23 0 02 Mar 2023
Grokking modular arithmetic Andrey Gromov 99 42 0 06 Jan 2023
Omnigrok: Grokking Beyond Algorithmic Data Ziming Liu Eric J. Michaud Max Tegmark 93 84 0 03 Oct 2022
Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limit Boaz Barak Benjamin L. Edelman Surbhi Goel Sham Kakade Eran Malach Cyril Zhang 108 133 0 18 Jul 2022
Random Feature Amplification: Feature Learning and Generalization in Neural Networks Spencer Frei Niladri S. Chatterji Peter L. Bartlett MLT 92 30 0 15 Feb 2022
A Farewell to the Bias-Variance Tradeoff? An Overview of the Theory of Overparameterized Machine Learning Yehuda Dar Vidya Muthukumar Richard G. Baraniuk 109 72 0 06 Sep 2021
Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolation M. Belkin 53 186 0 29 May 2021
Finite-sample Analysis of Interpolating Linear Classifiers in the Overparameterized Regime Niladri S. Chatterji Philip M. Long 86 109 0 25 Apr 2020
Surprises in High-Dimensional Ridgeless Least Squares Interpolation Trevor Hastie Andrea Montanari Saharon Rosset Robert Tibshirani 232 747 0 19 Mar 2019
Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernel Colin Wei Jason D. Lee Qiang Liu Tengyu Ma 252 245 0 12 Oct 2018