Deep Learning is Singular, and That's Good

22 October 2020

Papers citing "Deep Learning is Singular, and That's Good"

33 / 33 papers shown

Title
Review and Prospect of Algebraic Research in Equivalent Framework between Statistical Mechanics and Machine Learning Theory Sumio Watanabe 90 1 0 31 May 2024
Rethinking Parameter Counting in Deep Models: Effective Dimensionality Revisited Wesley J. Maddox Gregory W. Benton A. Wilson 128 61 0 04 Mar 2020
Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks Agustinus Kristiadi Matthias Hein Philipp Hennig BDL UQCV 88 289 0 24 Feb 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 645 4,921 0 23 Jan 2020
On the interplay between noise and curvature and its effect on optimization and generalization Valentin Thomas Fabian Pedregosa B. V. Merrienboer Pierre-Antoine Mangazol Yoshua Bengio Nicolas Le Roux 48 61 0 18 Jun 2019
Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networks Yuan Cao Quanquan Gu MLT AI4CE 105 391 0 30 May 2019
On the Power and Limitations of Random Features for Understanding Neural Networks Gilad Yehudai Ohad Shamir MLT 118 182 0 01 Apr 2019
Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks Sanjeev Arora S. Du Wei Hu Zhiyuan Li Ruosong Wang MLT 221 974 0 24 Jan 2019
Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers Zeyuan Allen-Zhu Yuanzhi Li Yingyu Liang MLT 201 775 0 12 Nov 2018
A Convergence Theory for Deep Learning via Over-Parameterization Zeyuan Allen-Zhu Yuanzhi Li Zhao Song AI4CE ODL 275 1,469 0 09 Nov 2018
Gradient Descent Provably Optimizes Over-parameterized Neural Networks S. Du Xiyu Zhai Barnabás Póczós Aarti Singh MLT ODL 238 1,276 0 04 Oct 2018
Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Data Yuanzhi Li Yingyu Liang MLT 222 653 0 03 Aug 2018
Stochastic natural gradient descent draws posterior samples in function space Samuel L. Smith Daniel Duckworth Semon Rezchikov Quoc V. Le Jascha Narain Sohl-Dickstein BDL 55 6 0 25 Jun 2018
Neural Tangent Kernel: Convergence and Generalization in Neural Networks Arthur Jacot Franck Gabriel Clément Hongler 275 3,223 0 20 Jun 2018
Energy-entropy competition and the effectiveness of stochastic gradient descent in machine learning Yao Zhang Andrew M. Saxe Madhu S. Advani A. Lee 69 60 0 05 Mar 2018
Stronger generalization bounds for deep nets via a compression approach Sanjeev Arora Rong Ge Behnam Neyshabur Yi Zhang MLT AI4CE 93 643 0 14 Feb 2018
Theory of Deep Learning III: explaining the non-overfitting puzzle T. Poggio Kenji Kawaguchi Q. Liao Brando Miranda Lorenzo Rosasco Xavier Boix Jack Hidary H. Mhaskar ODL 79 128 0 30 Dec 2017
Deep Learning Scaling is Predictable, Empirically Joel Hestness Sharan Narang Newsha Ardalani G. Diamos Heewoo Jun Hassan Kianinejad Md. Mostofa Ali Patwary Yang Yang Yanqi Zhou 112 744 0 01 Dec 2017
Three Factors Influencing Minima in SGD Stanislaw Jastrzebski Zachary Kenton Devansh Arpit Nicolas Ballas Asja Fischer Yoshua Bengio Amos Storkey 78 463 0 13 Nov 2017
SGD Learns Over-parameterized Networks that Provably Generalize on Linearly Separable Data Alon Brutzkus Amir Globerson Eran Malach Shai Shalev-Shwartz MLT 156 279 0 27 Oct 2017
A Bayesian Perspective on Generalization and Stochastic Gradient Descent Samuel L. Smith Quoc V. Le BDL 89 253 0 17 Oct 2017
Searching for Activation Functions Prajit Ramachandran Barret Zoph Quoc V. Le 87 612 0 16 Oct 2017
Exploring Generalization in Deep Learning Behnam Neyshabur Srinadh Bhojanapalli David A. McAllester Nathan Srebro FAtt 162 1,259 0 27 Jun 2017
Spectrally-normalized margin bounds for neural networks Peter L. Bartlett Dylan J. Foster Matus Telgarsky ODL 214 1,225 0 26 Jun 2017
Fractional Langevin Monte Carlo: Exploring Lévy Driven Stochastic Differential Equations for Markov Chain Monte Carlo Umut Simsekli 72 45 0 12 Jun 2017
Stochastic Gradient Descent as Approximate Bayesian Inference Stephan Mandt Matthew D. Hoffman David M. Blei BDL 67 599 0 13 Apr 2017
SGD Learns the Conjugate Kernel Class of the Network Amit Daniely 190 182 0 27 Feb 2017
Understanding deep learning requires rethinking generalization Chiyuan Zhang Samy Bengio Moritz Hardt Benjamin Recht Oriol Vinyals HAI 351 4,636 0 10 Nov 2016
Entropy-SGD: Biasing Gradient Descent Into Wide Valleys Pratik Chaudhari A. Choromańska Stefano Soatto Yann LeCun Carlo Baldassi C. Borgs J. Chayes Levent Sagun R. Zecchina ODL 96 775 0 06 Nov 2016
Gaussian Error Linear Units (GELUs) Dan Hendrycks Kevin Gimpel 174 5,049 0 27 Jun 2016
Norm-Based Capacity Control in Neural Networks Behnam Neyshabur Ryota Tomioka Nathan Srebro 292 591 0 27 Feb 2015
A Widely Applicable Bayesian Information Criterion Sumio Watanabe 103 787 0 31 Aug 2012
The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo Matthew D. Hoffman Andrew Gelman 189 4,315 0 18 Nov 2011