Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Data

3 August 2018

Papers citing "Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Data"

50 / 176 papers shown

Title
Generalized Leverage Score Sampling for Neural Networks Jason D. Lee Ruoqi Shen Zhao Song Mengdi Wang Zheng Yu 26 42 0 21 Sep 2020
Single-Timescale Actor-Critic Provably Finds Globally Optimal Policy Zuyue Fu Zhuoran Yang Zhaoran Wang 21 42 0 02 Aug 2020
Implicit Bias in Deep Linear Classification: Initialization Scale vs Training Accuracy E. Moroshko Suriya Gunasekar Blake E. Woodworth Jason D. Lee Nathan Srebro Daniel Soudry 35 85 0 13 Jul 2020
Weak error analysis for stochastic gradient descent optimization algorithms A. Bercher Lukas Gonon Arnulf Jentzen Diyora Salimova 36 4 0 03 Jul 2020
A Revision of Neural Tangent Kernel-based Approaches for Neural Networks Kyungsu Kim A. Lozano Eunho Yang AAML 40 0 0 02 Jul 2020
Training (Overparametrized) Neural Networks in Near-Linear Time Jan van den Brand Binghui Peng Zhao Song Omri Weinstein ODL 34 83 0 20 Jun 2020
Shape Matters: Understanding the Implicit Bias of the Noise Covariance Jeff Z. HaoChen Colin Wei Jason D. Lee Tengyu Ma 32 94 0 15 Jun 2020
Non-convergence of stochastic gradient descent in the training of deep neural networks Patrick Cheridito Arnulf Jentzen Florian Rossmannek 19 37 0 12 Jun 2020
Can Temporal-Difference and Q-Learning Learn Representation? A Mean-Field Theory Yufeng Zhang Qi Cai Zhuoran Yang Yongxin Chen Zhaoran Wang OOD MLT 159 11 0 08 Jun 2020
Feature Purification: How Adversarial Training Performs Robust Deep Learning Zeyuan Allen-Zhu Yuanzhi Li MLT AAML 39 148 0 20 May 2020
Symmetry & critical points for a model shallow neural network Yossi Arjevani M. Field 36 13 0 23 Mar 2020
The large learning rate phase of deep learning: the catapult mechanism Aitor Lewkowycz Yasaman Bahri Ethan Dyer Jascha Narain Sohl-Dickstein Guy Gur-Ari ODL 159 236 0 04 Mar 2020
Convex Geometry and Duality of Over-parameterized Neural Networks Tolga Ergen Mert Pilanci MLT 47 54 0 25 Feb 2020
Convergence of End-to-End Training in Deep Unsupervised Contrastive Learning Zixin Wen SSL 21 2 0 17 Feb 2020
Over-parameterized Adversarial Training: An Analysis Overcoming the Curse of Dimensionality Yi Zhang Orestis Plevrakis S. Du Xingguo Li Zhao Song Sanjeev Arora 29 51 0 16 Feb 2020
Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Loss Lénaïc Chizat Francis R. Bach MLT 39 329 0 11 Feb 2020
Distribution Approximation and Statistical Estimation Guarantees of Generative Adversarial Networks Minshuo Chen Wenjing Liao H. Zha Tuo Zhao 26 15 0 10 Feb 2020
Memory capacity of neural networks with threshold and ReLU activations Roman Vershynin 31 21 0 20 Jan 2020
Landscape Connectivity and Dropout Stability of SGD Solutions for Over-parameterized Neural Networks A. Shevchenko Marco Mondelli 27 37 0 20 Dec 2019
Optimization for deep learning: theory and algorithms Ruoyu Sun ODL 30 168 0 19 Dec 2019
Neural Tangents: Fast and Easy Infinite Neural Networks in Python Roman Novak Lechao Xiao Jiri Hron Jaehoon Lee Alexander A. Alemi Jascha Narain Sohl-Dickstein S. Schoenholz 38 225 0 05 Dec 2019
Towards Understanding the Spectral Bias of Deep Learning Yuan Cao Zhiying Fang Yue Wu Ding-Xuan Zhou Quanquan Gu 41 215 0 03 Dec 2019
Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms Kaipeng Zhang Zhuoran Yang Tamer Basar 63 1,184 0 24 Nov 2019
Neural Contextual Bandits with UCB-based Exploration Dongruo Zhou Lihong Li Quanquan Gu 36 15 0 11 Nov 2019
Enhanced Convolutional Neural Tangent Kernels Zhiyuan Li Ruosong Wang Dingli Yu S. Du Wei Hu Ruslan Salakhutdinov Sanjeev Arora 24 131 0 03 Nov 2019
Capacity, Bandwidth, and Compositionality in Emergent Language Learning Cinjon Resnick Abhinav Gupta Jakob N. Foerster Andrew M. Dai Kyunghyun Cho 31 51 0 24 Oct 2019
The Local Elasticity of Neural Networks Hangfeng He Weijie J. Su 45 44 0 15 Oct 2019
Algorithm-Dependent Generalization Bounds for Overparameterized Deep Residual Networks Spencer Frei Yuan Cao Quanquan Gu ODL 14 31 0 07 Oct 2019
Harnessing the Power of Infinitely Wide Deep Nets on Small-data Tasks Sanjeev Arora S. Du Zhiyuan Li Ruslan Salakhutdinov Ruosong Wang Dingli Yu AAML 22 161 0 03 Oct 2019
Beyond Linearization: On Quadratic and Higher-Order Approximation of Wide Neural Networks Yu Bai Jason D. Lee 24 116 0 03 Oct 2019
How does topology influence gradient propagation and model performance of deep networks with DenseNet-type skip connections? Kartikeya Bhardwaj Guihong Li R. Marculescu 38 1 0 02 Oct 2019
Stochastic AUC Maximization with Deep Neural Networks Mingrui Liu Zhuoning Yuan Yiming Ying Tianbao Yang 27 103 0 28 Aug 2019
Theoretical Issues in Deep Networks: Approximation, Optimization and Generalization T. Poggio Andrzej Banburski Q. Liao ODL 40 161 0 25 Aug 2019
The generalization error of random features regression: Precise asymptotics and double descent curve Song Mei Andrea Montanari 62 625 0 14 Aug 2019
Hessian based analysis of SGD for Deep Nets: Dynamics and Generalization Xinyan Li Qilong Gu Yingxue Zhou Tiancong Chen A. Banerjee ODL 42 51 0 24 Jul 2019
Benign Overfitting in Linear Regression Peter L. Bartlett Philip M. Long Gábor Lugosi Alexander Tsigler MLT 8 763 0 26 Jun 2019
Neural Proximal/Trust Region Policy Optimization Attains Globally Optimal Policy Boyi Liu Qi Cai Zhuoran Yang Zhaoran Wang 30 108 0 25 Jun 2019
Kernel and Rich Regimes in Overparametrized Models Blake E. Woodworth Suriya Gunasekar Pedro H. P. Savarese E. Moroshko Itay Golan Jason D. Lee Daniel Soudry Nathan Srebro 30 353 0 13 Jun 2019
Parameterized Structured Pruning for Deep Neural Networks Günther Schindler Wolfgang Roth Franz Pernkopf Holger Froening 24 6 0 12 Jun 2019
Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networks Yuan Cao Quanquan Gu MLT AI4CE 37 383 0 30 May 2019
Overlearning Reveals Sensitive Attributes Congzheng Song Vitaly Shmatikov 19 148 0 28 May 2019
SGD on Neural Networks Learns Functions of Increasing Complexity Preetum Nakkiran Gal Kaplun Dimitris Kalimeris Tristan Yang Benjamin L. Edelman Fred Zhang Boaz Barak MLT 28 236 0 28 May 2019
Fast Convergence of Natural Gradient Descent for Overparameterized Neural Networks Guodong Zhang James Martens Roger C. Grosse ODL 22 124 0 27 May 2019
Enhancing Adversarial Defense by k-Winners-Take-All Chang Xiao Peilin Zhong Changxi Zheng AAML 24 97 0 25 May 2019
What Can ResNet Learn Efficiently, Going Beyond Kernels? Zeyuan Allen-Zhu Yuanzhi Li 24 183 0 24 May 2019
Neural Temporal-Difference and Q-Learning Provably Converge to Global Optima Qi Cai Zhuoran Yang Jason D. Lee Zhaoran Wang 42 29 0 24 May 2019
Gradient Descent can Learn Less Over-parameterized Two-layer Neural Networks on Classification Problems Atsushi Nitanda Geoffrey Chinot Taiji Suzuki MLT 16 33 0 23 May 2019
How degenerate is the parametrization of neural networks with the ReLU activation function? Julius Berner Dennis Elbrächter Philipp Grohs ODL 33 28 0 23 May 2019
Data-dependent Sample Complexity of Deep Neural Networks via Lipschitz Augmentation Colin Wei Tengyu Ma 25 109 0 09 May 2019
On Exact Computation with an Infinitely Wide Neural Net Sanjeev Arora S. Du Wei Hu Zhiyuan Li Ruslan Salakhutdinov Ruosong Wang 65 906 0 26 Apr 2019