v1v2v3v4 (latest)

Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernel

12 October 2018

Papers citing "Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernel"

50 / 192 papers shown

Title
Weighted Neural Tangent Kernel: A Generalized and Improved Network-Induced Kernel Lei Tan Shutong Wu Xiaolin Huang 30 2 0 22 Mar 2021
Inductive Bias of Multi-Channel Linear Convolutional Networks with Bounded Weight Norm Meena Jagadeesan Ilya P. Razenshteyn Suriya Gunasekar 113 21 0 24 Feb 2021
Classifying high-dimensional Gaussian mixtures: Where kernel methods fail and neural networks succeed Maria Refinetti Sebastian Goldt Florent Krzakala Lenka Zdeborová 92 74 0 23 Feb 2021
Approximation and Learning with Deep Convolutional Models: a Kernel Perspective A. Bietti 89 30 0 19 Feb 2021
Bridging the Gap Between Adversarial Robustness and Optimization Bias Fartash Faghri Sven Gowal C. N. Vasconcelos David J. Fleet Fabian Pedregosa Nicolas Le Roux AAML 234 7 0 17 Feb 2021
A Local Convergence Theory for Mildly Over-Parameterized Two-Layer Neural Network Mo Zhou Rong Ge Chi Jin 145 46 0 04 Feb 2021
Implicit Bias of Linear RNNs M Motavali Emami Mojtaba Sahraee-Ardakan Parthe Pandit S. Rangan A. Fletcher 44 11 0 19 Jan 2021
Towards Understanding Learning in Neural Networks with Linear Teachers Roei Sarussi Alon Brutzkus Amir Globerson FedML MLT 122 21 0 07 Jan 2021
Provable Generalization of SGD-trained Neural Networks of Any Width in the Presence of Adversarial Label Noise Spencer Frei Yuan Cao Quanquan Gu FedML MLT 168 21 0 04 Jan 2021
Particle Dual Averaging: Optimization of Mean Field Neural Networks with Global Convergence Rate Analysis Atsushi Nitanda Denny Wu Taiji Suzuki 97 29 0 31 Dec 2020
Mathematical Models of Overparameterized Neural Networks Cong Fang Hanze Dong Tong Zhang 181 23 0 27 Dec 2020
Recent advances in deep learning theory Fengxiang He Dacheng Tao AI4CE 132 51 0 20 Dec 2020
Why Do Better Loss Functions Lead to Less Transferable Features? Simon Kornblith Ting-Li Chen Honglak Lee Mohammad Norouzi FaML 128 92 0 30 Oct 2020
Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depth Thao Nguyen M. Raghu Simon Kornblith OOD 107 284 0 29 Oct 2020
On Convergence and Generalization of Dropout Training Poorya Mianjy R. Arora 132 30 0 23 Oct 2020
Train simultaneously, generalize better: Stability of gradient-based minimax learners Farzan Farnia Asuman Ozdaglar 73 48 0 23 Oct 2020
Global optimality of softmax policy gradient with single hidden layer neural networks in the mean-field regime Andrea Agazzi Jianfeng Lu 91 16 0 22 Oct 2020
Label-Aware Neural Tangent Kernel: Toward Better Generalization and Local Elasticity Shuxiao Chen Hangfeng He Weijie J. Su 53 24 0 22 Oct 2020
Beyond Lazy Training for Over-parameterized Tensor Decomposition Xiang Wang Chenwei Wu Jason D. Lee Tengyu Ma Rong Ge 91 14 0 22 Oct 2020
Dual Averaging is Surprisingly Effective for Deep Learning Optimization Samy Jelassi Aaron Defazio 61 5 0 20 Oct 2020
Why Are Convolutional Nets More Sample-Efficient than Fully-Connected Nets? Zhiyuan Li Yi Zhang Sanjeev Arora BDL MLT 85 39 0 16 Oct 2020
A Modular Analysis of Provable Acceleration via Polyak's Momentum: Training a Wide ReLU Network and a Deep Linear Network Jun-Kun Wang Chi-Heng Lin Jacob D. Abernethy 80 24 0 04 Oct 2020
Obtaining Adjustable Regularization for Free via Iterate Averaging Jingfeng Wu Vladimir Braverman Lin F. Yang 63 2 0 15 Aug 2020
Multiple Descent: Design Your Own Generalization Curve Lin Chen Yifei Min M. Belkin Amin Karbasi DRL 162 61 0 03 Aug 2020
Finite Versus Infinite Neural Networks: an Empirical Study Jaehoon Lee S. Schoenholz Jeffrey Pennington Ben Adlam Lechao Xiao Roman Novak Jascha Narain Sohl-Dickstein 87 214 0 31 Jul 2020
Understanding Implicit Regularization in Over-Parameterized Single Index Model Jianqing Fan Zhuoran Yang Mengxin Yu 81 18 0 16 Jul 2020
Learning Over-Parametrized Two-Layer ReLU Neural Networks beyond NTK Yuanzhi Li Tengyu Ma Hongyang R. Zhang MLT 95 27 0 09 Jul 2020
Regularization Matters: A Nonparametric Perspective on Overparametrized Neural Network Tianyang Hu Wei Cao Cong Lin Guang Cheng 118 52 0 06 Jul 2020
Modeling from Features: a Mean-field Framework for Over-parameterized Deep Neural Networks Cong Fang Jason D. Lee Pengkun Yang Tong Zhang OOD FedML 156 58 0 03 Jul 2020
Towards Understanding Hierarchical Learning: Benefits of Neural Representations Minshuo Chen Yu Bai Jason D. Lee T. Zhao Huan Wang Caiming Xiong R. Socher SSL 95 49 0 24 Jun 2020
Generalisation Guarantees for Continual Learning with Orthogonal Gradient Descent Mehdi Abbana Bennani Thang Doan Masashi Sugiyama CLL 135 65 0 21 Jun 2020
Gradient descent follows the regularization path for general losses Ziwei Ji Miroslav Dudík Robert Schapire Matus Telgarsky AI4CE FaML 167 62 0 19 Jun 2020
Shape Matters: Understanding the Implicit Bias of the Noise Covariance Jeff Z. HaoChen Colin Wei Jason D. Lee Tengyu Ma 219 95 0 15 Jun 2020
On the training dynamics of deep networks with $L_2$ regularization Aitor Lewkowycz Guy Gur-Ari 113 54 0 15 Jun 2020
Directional convergence and alignment in deep learning Ziwei Ji Matus Telgarsky 73 171 0 11 Jun 2020
Banach Space Representer Theorems for Neural Networks and Ridge Splines Rahul Parhi Robert D. Nowak 35 7 0 10 Jun 2020
Can Temporal-Difference and Q-Learning Learn Representation? A Mean-Field Theory Yufeng Zhang Qi Cai Zhuoran Yang Yongxin Chen Zhaoran Wang OOD MLT 360 11 0 08 Jun 2020
Momentum-based variance-reduced proximal stochastic gradient method for composite nonconvex stochastic optimization Yangyang Xu Yibo Xu 69 25 0 31 May 2020
Provable Training of a ReLU Gate with an Iterative Non-Gradient Algorithm Sayar Karmakar Anirbit Mukherjee 52 7 0 08 May 2020
Optimization in Machine Learning: A Distribution Space Approach Yongqiang Cai Qianxiao Li Zuowei Shen 32 1 0 18 Apr 2020
A function space analysis of finite neural networks with insights from sampling theory Raja Giryes 72 6 0 15 Apr 2020
Adversarial Robustness Guarantees for Random Deep Neural Networks Giacomo De Palma B. Kiani S. Lloyd AAML OOD 55 8 0 13 Apr 2020
A Mean-field Analysis of Deep ResNet and Beyond: Towards Provable Optimization Via Overparameterization From Depth Yiping Lu Chao Ma Yulong Lu Jianfeng Lu Lexing Ying MLT 167 79 0 11 Mar 2020
Optimal Regularization Can Mitigate Double Descent Preetum Nakkiran Prayaag Venkat Sham Kakade Tengyu Ma 85 133 0 04 Mar 2020
Convex Geometry and Duality of Over-parameterized Neural Networks Tolga Ergen Mert Pilanci MLT 138 56 0 25 Feb 2020
An Optimization and Generalization Analysis for Max-Pooling Networks Alon Brutzkus Amir Globerson MLT AI4CE 46 4 0 22 Feb 2020
Revealing the Structure of Deep Neural Networks via Convex Duality Tolga Ergen Mert Pilanci MLT 105 72 0 22 Feb 2020
Few-Shot Learning via Learning the Representation, Provably S. Du Wei Hu Sham Kakade Jason D. Lee Qi Lei SSL 100 262 0 21 Feb 2020
Deep regularization and direct training of the inner layers of Neural Networks with Kernel Flows G. Yoo H. Owhadi 74 21 0 19 Feb 2020
Stochasticity of Deterministic Gradient Descent: Large Learning Rate for Multiscale Objective Function Lingkai Kong Molei Tao 57 23 0 14 Feb 2020