Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers

12 November 2018

Papers citing "Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers"

50 / 498 papers shown

Title
The Effects of Mild Over-parameterization on the Optimization Landscape of Shallow ReLU Neural Networks Itay Safran Gilad Yehudai Ohad Shamir 92 34 0 01 Jun 2020
Agnostic Learning of a Single Neuron with Gradient Descent Spencer Frei Yuan Cao Quanquan Gu MLT 20 59 0 29 May 2020
Feature Purification: How Adversarial Training Performs Robust Deep Learning Zeyuan Allen-Zhu Yuanzhi Li MLT AAML 37 147 0 20 May 2020
Joint Progressive Knowledge Distillation and Unsupervised Domain Adaptation Le Thanh Nguyen-Meidine Eric Granger M. Kiran Jose Dolz Louis-Antoine Blais-Morin 22 23 0 16 May 2020
Learning the gravitational force law and other analytic functions Atish Agarwala Abhimanyu Das Rina Panigrahy Qiuyi Zhang MLT 16 0 0 15 May 2020
Provable Training of a ReLU Gate with an Iterative Non-Gradient Algorithm Sayar Karmakar Anirbit Mukherjee 14 7 0 08 May 2020
Generalization Error of Generalized Linear Models in High Dimensions M Motavali Emami Mojtaba Sahraee-Ardakan Parthe Pandit S. Rangan A. Fletcher AI4CE 19 37 0 01 May 2020
Random Features for Kernel Approximation: A Survey on Algorithms, Theory, and Beyond Fanghui Liu Xiaolin Huang Yudong Chen Johan A. K. Suykens BDL 44 172 0 23 Apr 2020
Gradients as Features for Deep Representation Learning Fangzhou Mu Yingyu Liang Yin Li OOD 13 37 0 12 Apr 2020
Blind Adversarial Pruning: Balance Accuracy, Efficiency and Robustness Haidong Xie Lixin Qian Xueshuang Xiang Naijin Liu AAML 20 1 0 10 Apr 2020
Backprojection for Training Feedforward Neural Networks in the Input and Feature Spaces Benyamin Ghojogh Fakhri Karray Mark Crowley 12 2 0 05 Apr 2020
Memorizing Gaussians with no over-parameterizaion via gradient decent on neural networks Amit Daniely VLM MLT 10 15 0 28 Mar 2020
Piecewise linear activations substantially shape the loss surfaces of neural networks Fengxiang He Bohan Wang Dacheng Tao ODL 36 28 0 27 Mar 2020
Dimension Independent Generalization Error by Stochastic Gradient Descent Xi Chen Qiang Liu Xin T. Tong 13 1 0 25 Mar 2020
Neural Kernels Without Tangents Vaishaal Shankar Alex Fang Wenshuo Guo Sara Fridovich-Keil Ludwig Schmidt Jonathan Ragan-Kelley Benjamin Recht 25 90 0 04 Mar 2020
Semiparametric Nonlinear Bipartite Graph Representation Learning with Provable Guarantees Sen Na Yuwei Luo Zhuoran Yang Zhaoran Wang Mladen Kolar 17 7 0 02 Mar 2020
Convolutional Spectral Kernel Learning Jian Li Yong Liu Weiping Wang BDL 12 5 0 28 Feb 2020
Avoiding Kernel Fixed Points: Computing with ELU and GELU Infinite Networks Russell Tsuchida Tim Pearce Christopher van der Heide Fred Roosta M. Gallagher 8 8 0 20 Feb 2020
Learning Parities with Neural Networks Amit Daniely Eran Malach 24 76 0 18 Feb 2020
Convergence of End-to-End Training in Deep Unsupervised Contrastive Learning Zixin Wen SSL 21 2 0 17 Feb 2020
Over-parameterized Adversarial Training: An Analysis Overcoming the Curse of Dimensionality Yi Zhang Orestis Plevrakis S. Du Xingguo Li Zhao Song Sanjeev Arora 29 51 0 16 Feb 2020
Why Do Deep Residual Networks Generalize Better than Deep Feedforward Networks? -- A Neural Tangent Kernel Perspective Kaixuan Huang Yuqing Wang Molei Tao T. Zhao MLT 14 97 0 14 Feb 2020
Stochasticity of Deterministic Gradient Descent: Large Learning Rate for Multiscale Objective Function Lingkai Kong Molei Tao 12 22 0 14 Feb 2020
A Generalized Neural Tangent Kernel Analysis for Two-layer Neural Networks Zixiang Chen Yuan Cao Quanquan Gu Tong Zhang MLT 35 10 0 10 Feb 2020
Taylorized Training: Towards Better Approximation of Neural Network Training at Finite Width Yu Bai Ben Krause Huan Wang Caiming Xiong R. Socher 22 22 0 10 Feb 2020
Distribution Approximation and Statistical Estimation Guarantees of Generative Adversarial Networks Minshuo Chen Wenjing Liao H. Zha Tuo Zhao 26 15 0 10 Feb 2020
Quasi-Equivalence of Width and Depth of Neural Networks Fenglei Fan Rongjie Lai Ge Wang 22 11 0 06 Feb 2020
Function approximation by neural nets in the mean-field regime: Entropic regularization and controlled McKean-Vlasov dynamics Belinda Tzen Maxim Raginsky 18 17 0 05 Feb 2020
Proving the Lottery Ticket Hypothesis: Pruning is All You Need Eran Malach Gilad Yehudai Shai Shalev-Shwartz Ohad Shamir 64 271 0 03 Feb 2020
Identifying Mislabeled Data using the Area Under the Margin Ranking Geoff Pleiss Tianyi Zhang Ethan R. Elenberg Kilian Q. Weinberger NoLa 43 261 0 28 Jan 2020
Any Target Function Exists in a Neighborhood of Any Sufficiently Wide Random Network: A Geometrical Perspective S. Amari 27 12 0 20 Jan 2020
Big-Data Science in Porous Materials: Materials Genomics and Machine Learning Kevin Maik Jablonka D. Ongari S. M. Moosavi B. Smit AI4CE 31 350 0 18 Jan 2020
Learning a Single Neuron with Gradient Methods Gilad Yehudai Ohad Shamir MLT 11 63 0 15 Jan 2020
On Iterative Neural Network Pruning, Reinitialization, and the Similarity of Masks Michela Paganini Jessica Zosa Forde 19 19 0 14 Jan 2020
Deep Network Approximation for Smooth Functions Jianfeng Lu Zuowei Shen Haizhao Yang Shijun Zhang 67 247 0 09 Jan 2020
Landscape Connectivity and Dropout Stability of SGD Solutions for Over-parameterized Neural Networks A. Shevchenko Marco Mondelli 27 37 0 20 Dec 2019
A Finite-Time Analysis of Q-Learning with Neural Network Function Approximation Pan Xu Quanquan Gu 23 66 0 10 Dec 2019
How Much Over-parameterization Is Sufficient to Learn Deep ReLU Networks? Zixiang Chen Yuan Cao Difan Zou Quanquan Gu 22 122 0 27 Nov 2019
Convex Formulation of Overparameterized Deep Neural Networks Cong Fang Yihong Gu Weizhong Zhang Tong Zhang 34 28 0 18 Nov 2019
Tight Sample Complexity of Learning One-hidden-layer Convolutional Neural Networks Yuan Cao Quanquan Gu MLT 20 19 0 12 Nov 2019
Learning Boolean Circuits with Neural Networks Eran Malach Shai Shalev-Shwartz 9 4 0 25 Oct 2019
Over Parameterized Two-level Neural Networks Can Learn Near Optimal Feature Representations Cong Fang Hanze Dong Tong Zhang 17 18 0 25 Oct 2019
$Policy Optimization for $\mathcal{H}_2$ Linear Control with $\mathcal{H}_\infty$ Robustness Guarantee: Implicit Regularization and Global Convergence$ Policy Optimization for $\mathcal{H}_2$ Linear Control with $\mathcal{H}_\infty$ Robustness Guarantee: Implicit Regularization and Global Convergence Kaipeng Zhang Bin Hu Tamer Basar 24 119 0 21 Oct 2019
Neural tangent kernels, transportation mappings, and universal approximation Ziwei Ji Matus Telgarsky Ruicheng Xian 16 39 0 15 Oct 2019
The Local Elasticity of Neural Networks Hangfeng He Weijie J. Su 40 44 0 15 Oct 2019
Machine Learning for Prediction with Missing Dynamics J. Harlim Shixiao W. Jiang Senwei Liang Haizhao Yang AI4CE 14 60 0 13 Oct 2019
Nearly Minimal Over-Parametrization of Shallow Neural Networks Armin Eftekhari Chaehwan Song V. Cevher 21 1 0 09 Oct 2019
Harnessing the Power of Infinitely Wide Deep Nets on Small-data Tasks Sanjeev Arora S. Du Zhiyuan Li Ruslan Salakhutdinov Ruosong Wang Dingli Yu AAML 19 161 0 03 Oct 2019
Beyond Linearization: On Quadratic and Higher-Order Approximation of Wide Neural Networks Yu Bai J. Lee 24 116 0 03 Oct 2019
Environmental drivers of systematicity and generalization in a situated agent Felix Hill Andrew Kyle Lampinen R. Schneider S. Clark M. Botvinick James L. McClelland Adam Santoro OOD 9 103 0 01 Oct 2019