Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernel

12 October 2018

Papers citing "Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernel"

50 / 75 papers shown

Title
Embedding principle of homogeneous neural network for classification problem Jiahan Zhang Tao Luo Yaoyu Zhang 0 0 0 18 May 2025
LENSLLM: Unveiling Fine-Tuning Dynamics for LLM Selection Xinyue Zeng Haohui Wang Junhong Lin Jun Wu Tyler Cody Dawei Zhou 100 0 0 01 May 2025
Robust Feature Learning for Multi-Index Models in High Dimensions Alireza Mousavi-Hosseini Adel Javanmard Murat A. Erdogdu OOD AAML 44 1 0 21 Oct 2024
Sharper Guarantees for Learning Neural Network Classifiers with Gradient Methods Hossein Taheri Christos Thrampoulidis Arya Mazumdar MLT 36 0 0 13 Oct 2024
COOL: Efficient and Reliable Chain-Oriented Objective Logic with Neural Networks Feedback Control for Program Synthesis Jipeng Han 39 0 0 02 Oct 2024
Learning Multi-Index Models with Neural Networks via Mean-Field Langevin Dynamics Alireza Mousavi-Hosseini Denny Wu Murat A. Erdogdu MLT AI4CE 35 6 0 14 Aug 2024
Learning with Norm Constrained, Over-parameterized, Two-layer Neural Networks Fanghui Liu L. Dadi V. Cevher 82 2 0 29 Apr 2024
NTK-Guided Few-Shot Class Incremental Learning Jingren Liu Zhong Ji Yanwei Pang Yunlong Yu CLL 39 3 0 19 Mar 2024
Benign Overfitting and Grokking in ReLU Networks for XOR Cluster Data Zhiwei Xu Yutong Wang Spencer Frei Gal Vardi Wei Hu MLT 28 24 0 04 Oct 2023
How Over-Parameterization Slows Down Gradient Descent in Matrix Sensing: The Curses of Symmetry and Initialization Nuoya Xiong Lijun Ding Simon S. Du 35 11 0 03 Oct 2023
Gradient-Based Feature Learning under Structured Data Alireza Mousavi-Hosseini Denny Wu Taiji Suzuki Murat A. Erdogdu MLT 37 18 0 07 Sep 2023
Pareto Frontiers in Neural Feature Learning: Data, Compute, Width, and Luck Benjamin L. Edelman Surbhi Goel Sham Kakade Eran Malach Cyril Zhang 48 8 0 07 Sep 2023
Sharpness Minimization Algorithms Do Not Only Minimize Sharpness To Achieve Better Generalization Kaiyue Wen Zhiyuan Li Tengyu Ma FAtt 38 26 0 20 Jul 2023
Provable Guarantees for Nonlinear Feature Learning in Three-Layer Neural Networks Eshaan Nichani Alexandru Damian Jason D. Lee MLT 44 13 0 11 May 2023
Practically Solving LPN in High Noise Regimes Faster Using Neural Networks Haozhe Jiang Kaiyue Wen Yi-Long Chen 35 0 0 14 Mar 2023
Over-Parameterization Exponentially Slows Down Gradient Descent for Learning a Single Neuron Weihang Xu S. Du 37 16 0 20 Feb 2023
Birth-death dynamics for sampling: Global convergence, approximations and their asymptotics Yulong Lu D. Slepčev Lihan Wang 40 22 0 01 Nov 2022
A Functional-Space Mean-Field Theory of Partially-Trained Three-Layer Neural Networks Zhengdao Chen Eric Vanden-Eijnden Joan Bruna MLT 25 5 0 28 Oct 2022
Global Convergence of SGD On Two Layer Neural Nets Pulkit Gopalani Anirbit Mukherjee 26 5 0 20 Oct 2022
Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limit Boaz Barak Benjamin L. Edelman Surbhi Goel Sham Kakade Eran Malach Cyril Zhang 39 123 0 18 Jul 2022
Informed Learning by Wide Neural Networks: Convergence, Generalization and Sampling Complexity Jianyi Yang Shaolei Ren 32 3 0 02 Jul 2022
Neural Networks can Learn Representations with Gradient Descent Alexandru Damian Jason D. Lee Mahdi Soltanolkotabi SSL MLT 22 114 0 30 Jun 2022
$Intrinsic dimensionality and generalization properties of the $\mathcal{R}$-norm inductive bias$ Intrinsic dimensionality and generalization properties of the $\mathcal{R}$ -norm inductive bias Navid Ardeshir Daniel J. Hsu Clayton Sanford CML AI4CE 18 6 0 10 Jun 2022
Identifying good directions to escape the NTK regime and efficiently learn low-degree plus sparse polynomials Eshaan Nichani Yunzhi Bai Jason D. Lee 29 10 0 08 Jun 2022
Regularization-wise double descent: Why it occurs and how to eliminate it Fatih Yilmaz Reinhard Heckel 30 11 0 03 Jun 2022
High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representation Jimmy Ba Murat A. Erdogdu Taiji Suzuki Zhichao Wang Denny Wu Greg Yang MLT 40 121 0 03 May 2022
On Feature Learning in Neural Networks with Global Convergence Guarantees Zhengdao Chen Eric Vanden-Eijnden Joan Bruna MLT 36 13 0 22 Apr 2022
Sparse Neural Additive Model: Interpretable Deep Learning with Feature Selection via Group Sparsity Shiyun Xu Zhiqi Bu Pratik Chaudhari Ian Barnett 24 21 0 25 Feb 2022
Random Feature Amplification: Feature Learning and Generalization in Neural Networks Spencer Frei Niladri S. Chatterji Peter L. Bartlett MLT 30 29 0 15 Feb 2022
Convex Analysis of the Mean Field Langevin Dynamics Atsushi Nitanda Denny Wu Taiji Suzuki MLT 68 64 0 25 Jan 2022
DR3: Value-Based Deep Reinforcement Learning Requires Explicit Regularization Aviral Kumar Rishabh Agarwal Tengyu Ma Aaron Courville George Tucker Sergey Levine OffRL 31 65 0 09 Dec 2021
On the Equivalence between Neural Network and Support Vector Machine Yilan Chen Wei Huang Lam M. Nguyen Tsui-Wei Weng AAML 22 18 0 11 Nov 2021
Rethinking Neural vs. Matrix-Factorization Collaborative Filtering: the Theoretical Perspectives Zida Cheng Chuanwei Ruan Siheng Chen Sushant Kumar Ya-Qin Zhang 22 16 0 23 Oct 2021
Provable Regret Bounds for Deep Online Learning and Control Xinyi Chen Edgar Minasyan Jason D. Lee Elad Hazan 36 6 0 15 Oct 2021
Self-supervised Learning is More Robust to Dataset Imbalance Hong Liu Jeff Z. HaoChen Adrien Gaidon Tengyu Ma OOD SSL 33 157 0 11 Oct 2021
Tighter Sparse Approximation Bounds for ReLU Neural Networks Carles Domingo-Enrich Youssef Mroueh 99 4 0 07 Oct 2021
On the Global Convergence of Gradient Descent for multi-layer ResNets in the mean-field regime Zhiyan Ding Shi Chen Qin Li S. Wright MLT AI4CE 41 11 0 06 Oct 2021
The Limitations of Large Width in Neural Networks: A Deep Gaussian Process Perspective Geoff Pleiss John P. Cunningham 28 24 0 11 Jun 2021
Global Convergence of Three-layer Neural Networks in the Mean Field Regime H. Pham Phan-Minh Nguyen MLT AI4CE 41 19 0 11 May 2021
On Energy-Based Models with Overparametrized Shallow Neural Networks Carles Domingo-Enrich A. Bietti Eric Vanden-Eijnden Joan Bruna BDL 33 9 0 15 Apr 2021
Understanding the role of importance weighting for deep learning Da Xu Yuting Ye Chuanwei Ruan FAtt 36 43 0 28 Mar 2021
A Local Convergence Theory for Mildly Over-Parameterized Two-Layer Neural Network Mo Zhou Rong Ge Chi Jin 74 44 0 04 Feb 2021
Global optimality of softmax policy gradient with single hidden layer neural networks in the mean-field regime Andrea Agazzi Jianfeng Lu 13 15 0 22 Oct 2020
Why Are Convolutional Nets More Sample-Efficient than Fully-Connected Nets? Zhiyuan Li Yi Zhang Sanjeev Arora BDL MLT 6 39 0 16 Oct 2020
Obtaining Adjustable Regularization for Free via Iterate Averaging Jingfeng Wu Vladimir Braverman Lin F. Yang 30 2 0 15 Aug 2020
Multiple Descent: Design Your Own Generalization Curve Lin Chen Yifei Min M. Belkin Amin Karbasi DRL 26 61 0 03 Aug 2020
Generalisation Guarantees for Continual Learning with Orthogonal Gradient Descent Mehdi Abbana Bennani Thang Doan Masashi Sugiyama CLL 50 61 0 21 Jun 2020
When Does Preconditioning Help or Hurt Generalization? S. Amari Jimmy Ba Roger C. Grosse Xuechen Li Atsushi Nitanda Taiji Suzuki Denny Wu Ji Xu 36 32 0 18 Jun 2020
Shape Matters: Understanding the Implicit Bias of the Noise Covariance Jeff Z. HaoChen Colin Wei J. Lee Tengyu Ma 29 93 0 15 Jun 2020
On the training dynamics of deep networks with $L_2$ regularization Aitor Lewkowycz Guy Gur-Ari 36 53 0 15 Jun 2020