Generalisation error in learning with random features and the hidden manifold model

21 February 2020

Lenka Zdeborová

Papers citing "Generalisation error in learning with random features and the hidden manifold model"

40 / 40 papers shown

Title
The Effect of Optimal Self-Distillation in Noisy Gaussian Mixture Model Kaito Takanami Takashi Takahashi Ayaka Sakata 65 1 0 27 Jan 2025
Bilinear Sequence Regression: A Model for Learning from Long Sequences of High-dimensional Tokens Vittorio Erba Emanuele Troiani Luca Biggio Antoine Maillard Lenka Zdeborová 91 1 0 24 Oct 2024
Asymptotic theory of in-context learning by linear attention Yue M. Lu Mary I. Letey Jacob A. Zavatone-Veth Anindita Maiti Cengiz Pehlevan 50 12 0 20 May 2024
Restoring balance: principled under/oversampling of data for optimal classification Emanuele Loffredo Mauro Pastore Simona Cocco R. Monasson 50 9 0 15 May 2024
The Gaussian equivalence of generative models for learning with shallow neural networks Sebastian Goldt Bruno Loureiro Galen Reeves Florent Krzakala M. Mézard Lenka Zdeborová BDL 50 101 0 25 Jun 2020
Random Matrix Theory Proves that Deep Learning Representations of GAN-data Behave as Gaussian Mixtures M. Seddik Cosme Louart M. Tamaazousti Romain Couillet 29 67 0 21 Jan 2020
Deep Double Descent: Where Bigger Models and More Data Hurt Preetum Nakkiran Gal Kaplun Yamini Bansal Tristan Yang Boaz Barak Ilya Sutskever 77 925 0 04 Dec 2019
Kernel computations from large-scale random features obtained by Optical Processing Units Ruben Ohana Jonas Wacker Jonathan Dong Sébastien Marmin Florent Krzakala Maurizio Filippone L. Daudet 24 24 0 22 Oct 2019
Modelling the influence of data structure on learning in neural networks: the hidden manifold model Sebastian Goldt M. Mézard Florent Krzakala Lenka Zdeborová BDL 46 51 0 25 Sep 2019
The generalization error of random features regression: Precise asymptotics and double descent curve Song Mei Andrea Montanari 66 631 0 14 Aug 2019
Disentangling feature and lazy training in deep neural networks Mario Geiger S. Spigler Arthur Jacot Matthieu Wyart 73 17 0 19 Jun 2019
Kernel and Rich Regimes in Overparametrized Models Blake E. Woodworth Suriya Gunasekar Pedro H. P. Savarese E. Moroshko Itay Golan Jason D. Lee Daniel Soudry Nathan Srebro 46 358 0 13 Jun 2019
Surprises in High-Dimensional Ridgeless Least Squares Interpolation Trevor Hastie Andrea Montanari Saharon Rosset Robert Tibshirani 83 737 0 19 Mar 2019
Scaling description of generalization with number of parameters in deep learning Mario Geiger Arthur Jacot S. Spigler Franck Gabriel Levent Sagun Stéphane dÁscoli Giulio Biroli Clément Hongler Matthieu Wyart 57 195 0 06 Jan 2019
Reconciling modern machine learning practice and the bias-variance trade-off M. Belkin Daniel J. Hsu Siyuan Ma Soumik Mandal 140 1,628 0 28 Dec 2018
On Lazy Training in Differentiable Programming Lénaïc Chizat Edouard Oyallon Francis R. Bach 72 823 0 19 Dec 2018
A Convergence Theory for Deep Learning via Over-Parameterization Zeyuan Allen-Zhu Yuanzhi Li Zhao Song AI4CE ODL 130 1,457 0 09 Nov 2018
A jamming transition from under- to over-parametrization affects loss landscape and generalization S. Spigler Mario Geiger Stéphane dÁscoli Levent Sagun Giulio Biroli Matthieu Wyart 42 152 0 22 Oct 2018
A Modern Take on the Bias-Variance Tradeoff in Neural Networks Brady Neal Sarthak Mittal A. Baratin Vinayak Tantia Matthew Scicluna Simon Lacoste-Julien Ioannis Mitliagkas 60 169 0 19 Oct 2018
Gradient Descent Provably Optimizes Over-parameterized Neural Networks S. Du Xiyu Zhai Barnabás Póczós Aarti Singh MLT ODL 108 1,261 0 04 Oct 2018
Neural Tangent Kernel: Convergence and Generalization in Neural Networks Arthur Jacot Franck Gabriel Clément Hongler 121 3,160 0 20 Jun 2018
The committee machine: Computational to statistical gaps in learning a two-layers neural network Benjamin Aubin Antoine Maillard Jean Barbier Florent Krzakala N. Macris Lenka Zdeborová 53 105 0 14 Jun 2018
The phase transition for the existence of the maximum likelihood estimate in high-dimensional logistic regression Emmanuel J. Candes Pragya Sur 29 140 0 25 Apr 2018
High-dimensional dynamics of generalization error in neural networks Madhu S. Advani Andrew M. Saxe AI4CE 106 467 0 10 Oct 2017
Optimal Errors and Phase Transitions in High-Dimensional Generalized Linear Models Jean Barbier Florent Krzakala N. Macris Léo Miolane Lenka Zdeborová 55 262 0 10 Aug 2017
Exploring Generalization in Deep Learning Behnam Neyshabur Srinadh Bhojanapalli David A. McAllester Nathan Srebro FAtt 121 1,245 0 27 Jun 2017
FALKON: An Optimal Large Scale Kernel Method Alessandro Rudi Luigi Carratino Lorenzo Rosasco 30 196 0 31 May 2017
The Unreasonable Effectiveness of Structured Random Orthogonal Embeddings K. Choromanski Mark Rowland Adrian Weller 57 85 0 02 Mar 2017
A Random Matrix Approach to Neural Networks Cosme Louart Zhenyu Liao Romain Couillet 30 161 0 17 Feb 2017
Understanding deep learning requires rethinking generalization Chiyuan Zhang Samy Bengio Moritz Hardt Benjamin Recht Oriol Vinyals HAI 215 4,612 0 10 Nov 2016
Structured adaptive and random spinners for fast machine learning computations Mariusz Bojarski A. Choromańska K. Choromanski Francois Fagan Cédric Gouy-Pailler Anne Morvan Nourhan Sakr Tamás Sarlós Jamal Atif 42 35 0 19 Oct 2016
ACDC: A Structured Efficient Linear Layer Marcin Moczulski Misha Denil J. Appleyard Nando de Freitas 52 98 0 18 Nov 2015
Statistical physics of inference: Thresholds and algorithms Lenka Zdeborová Florent Krzakala AI4CE 36 422 0 08 Nov 2015
Random Projections through multiple optical scattering: Approximating kernels at the speed of light Alaa Saade F. Caltagirone I. Carron L. Daudet Angélique Dremeau S. Gigan Florent Krzakala 24 117 0 22 Oct 2015
The Spectral Norm of Random Inner-Product Kernel Matrices Z. Fan Andrea Montanari 53 47 0 19 Jul 2015
Fastfood: Approximate Kernel Expansions in Loglinear Time Quoc V. Le Tamás Sarlós Alex Smola 42 442 0 13 Aug 2014
Auto-Encoding Variational Bayes Diederik P. Kingma Max Welling BDL 300 16,972 0 20 Dec 2013
Divide and Conquer Kernel Ridge Regression: A Distributed Algorithm with Minimax Optimal Rates Yuchen Zhang John C. Duchi Martin J. Wainwright 79 376 0 22 May 2013
Statistical mechanics of complex neural systems and high dimensional data Madhu S. Advani Subhaneil Lahiri Surya Ganguli AI4CE 50 70 0 30 Jan 2013
A typical reconstruction limit of compressed sensing based on Lp-norm minimization Y. Kabashima Tadashi Wadayama Toshiyuki Tanaka 59 159 0 06 Jul 2009