v1v2v3v4 (latest)

Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernel

12 October 2018

Papers citing "Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernel"

50 / 192 papers shown

Title
Global Convergence of SGD On Two Layer Neural Nets Pulkit Gopalani Anirbit Mukherjee 69 6 0 20 Oct 2022
The Asymmetric Maximum Margin Bias of Quasi-Homogeneous Neural Networks D. Kunin Atsushi Yamamura Chao Ma Surya Ganguli 79 21 0 07 Oct 2022
Fast Neural Kernel Embeddings for General Activations Insu Han A. Zandieh Jaehoon Lee Roman Novak Lechao Xiao Amin Karbasi 120 19 0 09 Sep 2022
On the non-universality of deep learning: quantifying the cost of symmetry Emmanuel Abbe Enric Boix-Adserà FedML MLT 80 19 0 05 Aug 2022
Feature selection with gradient descent on two-layer networks in low-rotation regimes Matus Telgarsky MLT 81 16 0 04 Aug 2022
Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limit Boaz Barak Benjamin L. Edelman Surbhi Goel Sham Kakade Eran Malach Cyril Zhang 114 133 0 18 Jul 2022
Informed Learning by Wide Neural Networks: Convergence, Generalization and Sampling Complexity Jianyi Yang Shaolei Ren 88 3 0 02 Jul 2022
Neural Networks can Learn Representations with Gradient Descent Alexandru Damian Jason D. Lee Mahdi Soltanolkotabi SSL MLT 102 123 0 30 Jun 2022
How You Start Matters for Generalization Sameera Ramasinghe L. MacDonald M. Farazi Hemanth Saratchandran Simon Lucey ODL AI4CE 89 6 0 17 Jun 2022
Max-Margin Works while Large Margin Fails: Generalization without Uniform Convergence Margalit Glasgow Colin Wei Mary Wootters Tengyu Ma 96 5 0 16 Jun 2022
$Intrinsic dimensionality and generalization properties of the $\mathcal{R}$-norm inductive bias$ Intrinsic dimensionality and generalization properties of the $\mathcal{R}$ -norm inductive bias Navid Ardeshir Daniel J. Hsu Clayton Sanford CML AI4CE 113 6 0 10 Jun 2022
Identifying good directions to escape the NTK regime and efficiently learn low-degree plus sparse polynomials Eshaan Nichani Yunzhi Bai Jason D. Lee 85 10 0 08 Jun 2022
Regularization-wise double descent: Why it occurs and how to eliminate it Fatih Yilmaz Reinhard Heckel 85 11 0 03 Jun 2022
Long-Tailed Learning Requires Feature Learning T. Laurent J. V. Brecht Xavier Bresson VLM 93 1 0 29 May 2022
Generalization Bounds for Gradient Methods via Discrete and Continuous Prior Jun Yu Li Xu Luo Jian Li 80 4 0 27 May 2022
Fast Instrument Learning with Faster Rates Ziyu Wang Yuhao Zhou Jun Zhu 109 3 0 22 May 2022
High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representation Jimmy Ba Murat A. Erdogdu Taiji Suzuki Zhichao Wang Denny Wu Greg Yang MLT 99 129 0 03 May 2022
On Feature Learning in Neural Networks with Global Convergence Guarantees Zhengdao Chen Eric Vanden-Eijnden Joan Bruna MLT 96 13 0 22 Apr 2022
High-dimensional Asymptotics of Langevin Dynamics in Spiked Matrix Models Tengyuan Liang Subhabrata Sen Pragya Sur 83 7 0 09 Apr 2022
Surrogate Gap Minimization Improves Sharpness-Aware Training Juntang Zhuang Boqing Gong Liangzhe Yuan Huayu Chen Hartwig Adam Nicha Dvornek S. Tatikonda James Duncan Ting Liu 107 158 0 15 Mar 2022
Sparse Neural Additive Model: Interpretable Deep Learning with Feature Selection via Group Sparsity Shiyun Xu Zhiqi Bu Pratik Chaudhari Ian Barnett 86 23 0 25 Feb 2022
Random Feature Amplification: Feature Learning and Generalization in Neural Networks Spencer Frei Niladri S. Chatterji Peter L. Bartlett MLT 103 30 0 15 Feb 2022
Convex Analysis of the Mean Field Langevin Dynamics Atsushi Nitanda Denny Wu Taiji Suzuki MLT 172 66 0 25 Jan 2022
Wasserstein Flow Meets Replicator Dynamics: A Mean-Field Analysis of Representation Learning in Actor-Critic Yufeng Zhang Siyu Chen Zhuoran Yang Michael I. Jordan Zhaoran Wang 128 4 0 27 Dec 2021
Integral representations of shallow neural network with Rectified Power Unit activation function Ahmed Abdeljawad Philipp Grohs 42 10 0 20 Dec 2021
DR3: Value-Based Deep Reinforcement Learning Requires Explicit Regularization Aviral Kumar Rishabh Agarwal Tengyu Ma Aaron Courville George Tucker Sergey Levine OffRL 97 69 0 09 Dec 2021
On the Equivalence between Neural Network and Support Vector Machine Yilan Chen Wei Huang Lam M. Nguyen Tsui-Wei Weng AAML 80 18 0 11 Nov 2021
Dynamics of Local Elasticity During Training of Neural Nets Soham Dan Anirbit Mukherjee Avirup Das Phanideep Gampa 73 0 0 01 Nov 2021
Limiting fluctuation and trajectorial stability of multilayer neural networks with mean field training H. Pham Phan-Minh Nguyen 73 6 0 29 Oct 2021
Rethinking Neural vs. Matrix-Factorization Collaborative Filtering: the Theoretical Perspectives Zida Cheng Chuanwei Ruan Siheng Chen Sushant Kumar Ya Zhang 84 16 0 23 Oct 2021
Provable Regret Bounds for Deep Online Learning and Control Xinyi Chen Edgar Minasyan Jason D. Lee Elad Hazan 115 6 0 15 Oct 2021
Self-supervised Learning is More Robust to Dataset Imbalance Hong Liu Jeff Z. HaoChen Adrien Gaidon Tengyu Ma OOD SSL 86 167 0 11 Oct 2021
Tighter Sparse Approximation Bounds for ReLU Neural Networks Carles Domingo-Enrich Youssef Mroueh 143 4 0 07 Oct 2021
On the Global Convergence of Gradient Descent for multi-layer ResNets in the mean-field regime Zhiyan Ding Shi Chen Qin Li S. Wright MLT AI4CE 102 11 0 06 Oct 2021
Statistically Meaningful Approximation: a Case Study on Approximating Turing Machines with Transformers Colin Wei Yining Chen Tengyu Ma 79 92 0 28 Jul 2021
Going Beyond Linear RL: Sample Efficient Neural Function Approximation Baihe Huang Kaixuan Huang Sham Kakade Jason D. Lee Qi Lei Runzhe Wang Jiaqi Yang 103 8 0 14 Jul 2021
Optimal Gradient-based Algorithms for Non-concave Bandit Optimization Baihe Huang Kaixuan Huang Sham Kakade Jason D. Lee Qi Lei Runzhe Wang Jiaqi Yang 417 17 0 09 Jul 2021
Understanding Deflation Process in Over-parametrized Tensor Decomposition Rong Ge Y. Ren Xiang Wang Mo Zhou 80 19 0 11 Jun 2021
The Limitations of Large Width in Neural Networks: A Deep Gaussian Process Perspective Geoff Pleiss John P. Cunningham 76 27 0 11 Jun 2021
Separation Results between Fixed-Kernel and Feature-Learning Probability Metrics Carles Domingo-Enrich Youssef Mroueh 66 1 0 10 Jun 2021
Overparameterization of deep ResNet: zero loss and mean-field analysis Zhiyan Ding Shi Chen Qin Li S. Wright ODL 95 25 0 30 May 2021
Properties of the After Kernel Philip M. Long 66 29 0 21 May 2021
Global Convergence of Three-layer Neural Networks in the Mean Field Regime H. Pham Phan-Minh Nguyen MLT AI4CE 91 19 0 11 May 2021
Achieving Small Test Error in Mildly Overparameterized Neural Networks Shiyu Liang Ruoyu Sun R. Srikant 61 3 0 24 Apr 2021
On Energy-Based Models with Overparametrized Shallow Neural Networks Carles Domingo-Enrich A. Bietti Eric Vanden-Eijnden Joan Bruna BDL 57 9 0 15 Apr 2021
An Adaptive Synaptic Array using Fowler-Nordheim Dynamic Analog Memory Darshit Mehta K. Aono S. Chakrabartty 40 12 0 13 Apr 2021
A Recipe for Global Convergence Guarantee in Deep Neural Networks Kenji Kawaguchi Qingyun Sun 58 12 0 12 Apr 2021
Understanding the role of importance weighting for deep learning Da Xu Yuting Ye Chuanwei Ruan FAtt 95 44 0 28 Mar 2021
Recent Advances in Large Margin Learning Yiwen Guo Changshui Zhang AAML AI4CE 121 13 0 25 Mar 2021
Why Do Local Methods Solve Nonconvex Problems? Tengyu Ma 54 13 0 24 Mar 2021