High-dimensional dynamics of generalization error in neural networks

10 October 2017

Papers citing "High-dimensional dynamics of generalization error in neural networks"

50 / 296 papers shown

Title
On the generalization of learning algorithms that do not converge N. Chandramoorthy Andreas Loukas Khashayar Gatmiry Stefanie Jegelka MLT 19 11 0 16 Aug 2022
Benign, Tempered, or Catastrophic: A Taxonomy of Overfitting Neil Rohit Mallinar James B. Simon Amirhesam Abedsoltan Parthe Pandit M. Belkin Preetum Nakkiran 24 37 0 14 Jul 2022
Studying Generalization Through Data Averaging C. Gomez-Uribe FedML 19 0 0 28 Jun 2022
How You Start Matters for Generalization Sameera Ramasinghe L. MacDonald M. Farazi Hemanth Saratchandran Simon Lucey ODL AI4CE 31 6 0 17 Jun 2022
Trajectory-dependent Generalization Bounds for Deep Neural Networks via Fractional Brownian Motion Chengli Tan Jiang Zhang Junmin Liu 35 1 0 09 Jun 2022
How does overparametrization affect performance on minority groups? Subha Maity Saptarshi Roy Songkai Xue Mikhail Yurochkin Yuekai Sun 23 3 0 07 Jun 2022
Few-Shot Learning by Dimensionality Reduction in Gradient Space M. Gauch M. Beck Thomas Adler D. Kotsur Stefan Fiel ... Markus Holzleitner Werner Zellinger D. Klotz Sepp Hochreiter Sebastian Lehner 43 9 0 07 Jun 2022
Beyond accuracy: generalization properties of bio-plausible temporal credit assignment rules Yuhan Helena Liu Arna Ghosh Blake A. Richards E. Shea-Brown Guillaume Lajoie 28 9 0 02 Jun 2022
Precise Learning Curves and Higher-Order Scaling Limits for Dot Product Kernel Regression Lechao Xiao Hong Hu Theodor Misiakiewicz Yue M. Lu Jeffrey Pennington 65 18 0 30 May 2022
Scaling Laws and Interpretability of Learning from Repeated Data Danny Hernandez Tom B. Brown Tom Conerly Nova Dassarma Dawn Drain ... Catherine Olsson Dario Amodei Nicholas Joseph Jared Kaplan Sam McCandlish 25 111 0 21 May 2022
Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networks Blake Bordelon C. Pehlevan MLT 34 78 0 19 May 2022
Ridgeless Regression with Random Features Jian Li Yong-Jin Liu Yingying Zhang 24 2 0 01 May 2022
Overparameterized Linear Regression under Adversarial Attacks Antônio H. Ribeiro Thomas B. Schon AAML 12 18 0 13 Apr 2022
Can Neural Nets Learn the Same Model Twice? Investigating Reproducibility and Double Descent from the Decision Boundary Perspective Gowthami Somepalli Liam H. Fowl Arpit Bansal Ping Yeh-Chiang Yehuda Dar Richard Baraniuk Micah Goldblum Tom Goldstein 16 64 0 15 Mar 2022
Phenomenology of Double Descent in Finite-Width Neural Networks Sidak Pal Singh Aurelien Lucchi Thomas Hofmann Bernhard Schölkopf 26 10 0 14 Mar 2022
Bias-variance decomposition of overparameterized regression with random linear features J. Rocks Pankaj Mehta 20 12 0 10 Mar 2022
Contrasting random and learned features in deep Bayesian linear regression Jacob A. Zavatone-Veth William L. Tong C. Pehlevan BDL MLT 28 26 0 01 Mar 2022
On the Origins of the Block Structure Phenomenon in Neural Network Representations Thao Nguyen M. Raghu Simon Kornblith 25 14 0 15 Feb 2022
Towards Data-Algorithm Dependent Generalization: a Case Study on Overparameterized Linear Regression Jing Xu Jiaye Teng Yang Yuan Andrew Chi-Chih Yao 23 1 0 12 Feb 2022
Investigating Power laws in Deep Representation Learning Arna Ghosh Arnab Kumar Mondal Kumar Krishna Agrawal Blake A. Richards SSL OOD 11 19 0 11 Feb 2022
Data-driven emergence of convolutional structure in neural networks Alessandro Ingrosso Sebastian Goldt 53 38 0 01 Feb 2022
Fluctuations, Bias, Variance & Ensemble of Learners: Exact Asymptotics for Convex Losses in High-Dimension Bruno Loureiro Cédric Gerbelot Maria Refinetti G. Sicuro Florent Krzakala 33 25 0 31 Jan 2022
Error Scaling Laws for Kernel Classification under Source and Capacity Conditions Hugo Cui Bruno Loureiro Florent Krzakala Lenka Zdeborová 46 10 0 29 Jan 2022
A phase transition for finding needles in nonlinear haystacks with LASSO artificial neural networks Xiaoyu Ma S. Sardy N. Hengartner Nikolai Bobenko Yen Ting Lin 27 2 0 21 Jan 2022
Largest Eigenvalues of the Conjugate Kernel of Single-Layered Neural Networks L. Benigni Sandrine Péché 36 8 0 13 Jan 2022
Complexity from Adaptive-Symmetries Breaking: Global Minima in the Statistical Mechanics of Deep Neural Networks Shaun Li AI4CE 38 0 0 03 Jan 2022
Multi-scale Feature Learning Dynamics: Insights for Double Descent Mohammad Pezeshki Amartya Mitra Yoshua Bengio Guillaume Lajoie 61 25 0 06 Dec 2021
KNAS: Green Neural Architecture Search Jingjing Xu Liang Zhao Junyang Lin Rundong Gao Xu Sun Hongxia Yang 17 56 0 26 Nov 2021
Model, sample, and epoch-wise descents: exact solution of gradient flow in the random feature model A. Bodin N. Macris 37 13 0 22 Oct 2021
Conditioning of Random Feature Matrices: Double Descent and Generalization Error Zhijun Chen Hayden Schaeffer 35 12 0 21 Oct 2021
Random matrices in service of ML footprint: ternary random features with no performance loss Hafiz Tiomoko Ali Zhenyu Liao Romain Couillet 44 7 0 05 Oct 2021
Stochastic Training is Not Necessary for Generalization Jonas Geiping Micah Goldblum Phillip E. Pope Michael Moeller Tom Goldstein 89 72 0 29 Sep 2021
Scaling Laws for Neural Machine Translation Behrooz Ghorbani Orhan Firat Markus Freitag Ankur Bapna M. Krikun Xavier Garcia Ciprian Chelba Colin Cherry 40 99 0 16 Sep 2021
A Farewell to the Bias-Variance Tradeoff? An Overview of the Theory of Overparameterized Machine Learning Yehuda Dar Vidya Muthukumar Richard G. Baraniuk 29 71 0 06 Sep 2021
When and how epochwise double descent happens Cory Stephenson Tyler Lee 34 15 0 26 Aug 2021
A Random Matrix Perspective on Random Tensors J. H. D. M. Goulart Romain Couillet P. Comon 19 22 0 02 Aug 2021
An Instance-Dependent Simulation Framework for Learning with Label Noise Keren Gu Xander Masotto Vandana Bachani Balaji Lakshminarayanan Jack Nikodem Dong Yin NoLa 11 24 0 23 Jul 2021
Edge of chaos as a guiding principle for modern neural network training Lin Zhang Ling Feng Kan Chen C. Lai 16 9 0 20 Jul 2021
Continuous vs. Discrete Optimization of Deep Neural Networks Omer Elkabetz Nadav Cohen 68 44 0 14 Jul 2021
The Foes of Neural Network's Data Efficiency Among Unnecessary Input Dimensions Vanessa D’Amario S. Srivastava Tomotake Sasaki Xavier Boix AAML 21 2 0 13 Jul 2021
Continual Learning in the Teacher-Student Setup: Impact of Task Similarity Sebastian Lee Sebastian Goldt Andrew M. Saxe CLL 29 73 0 09 Jul 2021
Mitigating deep double descent by concatenating inputs John Chen Qihan Wang Anastasios Kyrillidis BDL 8 3 0 02 Jul 2021
Saddle-to-Saddle Dynamics in Deep Linear Networks: Small Initialization Training, Symmetry, and Sparsity Arthur Jacot François Ged Berfin cSimcsek Clément Hongler Franck Gabriel 27 52 0 30 Jun 2021
On the Sample Complexity and Metastability of Heavy-tailed Policy Search in Continuous Control Amrit Singh Bedi Anjaly Parayil Junyu Zhang Mengdi Wang Alec Koppel 30 15 0 15 Jun 2021
An Analytical Theory of Curriculum Learning in Teacher-Student Networks Luca Saglietti Stefano Sarao Mannelli Andrew M. Saxe 22 25 0 15 Jun 2021
Probing transfer learning with a model of synthetic correlated datasets Federica Gerace Luca Saglietti Stefano Sarao Mannelli Andrew M. Saxe Lenka Zdeborová OOD 16 30 0 09 Jun 2021
Redundant representations help generalization in wide neural networks Diego Doimo Aldo Glielmo Sebastian Goldt A. Laio AI4CE 25 9 0 07 Jun 2021
Double Descent Optimization Pattern and Aliasing: Caveats of Noisy Labels Florian Dubost Erin Hong Max Pike Siddharth Sharma Siyi Tang Nandita Bhaskhar Christopher Lee-Messer D. Rubin NoLa 37 0 0 03 Jun 2021
Optimization Variance: Exploring Generalization Properties of DNNs Xiao Zhang Dongrui Wu Haoyi Xiong Bo Dai 15 4 0 03 Jun 2021
Generalization Error Rates in Kernel Regression: The Crossover from the Noiseless to Noisy Regime Hugo Cui Bruno Loureiro Florent Krzakala Lenka Zdeborová 32 82 0 31 May 2021