Theory of Deep Learning III: explaining the non-overfitting puzzle

30 December 2017

Papers citing "Theory of Deep Learning III: explaining the non-overfitting puzzle"

21 / 21 papers shown

Title
Functional Risk Minimization Ferran Alet Clement Gehring Tomás Lozano-Pérez Kenji Kawaguchi Joshua B. Tenenbaum Leslie Pack Kaelbling OffRL 60 0 0 31 Dec 2024
An SDE for Modeling SAM: Theory and Insights Enea Monzio Compagnoni Luca Biggio Antonio Orvieto F. Proske Hans Kersting Aurelien Lucchi 25 13 0 19 Jan 2023
WLD-Reg: A Data-dependent Within-layer Diversity Regularizer Firas Laakom Jenni Raitoharju Alexandros Iosifidis Moncef Gabbouj AI4CE 29 7 0 03 Jan 2023
PD-Quant: Post-Training Quantization based on Prediction Difference Metric Jiawei Liu Lin Niu Zhihang Yuan Dawei Yang Xinggang Wang Wenyu Liu MQ 98 69 0 14 Dec 2022
The Limiting Dynamics of SGD: Modified Loss, Phase Space Oscillations, and Anomalous Diffusion D. Kunin Javier Sagastuy-Breña Lauren Gillespie Eshed Margalit Hidenori Tanaka Surya Ganguli Daniel L. K. Yamins 31 15 0 19 Jul 2021
Learning distinct features helps, provably Firas Laakom Jenni Raitoharju Alexandros Iosifidis Moncef Gabbouj MLT 36 6 0 10 Jun 2021
Provable Super-Convergence with a Large Cyclical Learning Rate Samet Oymak 35 12 0 22 Feb 2021
Gradient Starvation: A Learning Proclivity in Neural Networks Mohammad Pezeshki Sekouba Kaba Yoshua Bengio Aaron Courville Doina Precup Guillaume Lajoie MLT 50 258 0 18 Nov 2020
Deep Learning is Singular, and That's Good Daniel Murfet Susan Wei Biwei Huang Hui Li Jesse Gell-Redman T. Quella UQCV 24 26 0 22 Oct 2020
An analytic theory of shallow networks dynamics for hinge loss classification Franco Pellegrini Giulio Biroli 35 19 0 19 Jun 2020
Shape Matters: Understanding the Implicit Bias of the Noise Covariance Jeff Z. HaoChen Colin Wei J. Lee Tengyu Ma 32 94 0 15 Jun 2020
Deep Gamblers: Learning to Abstain with Portfolio Theory Liu Ziyin Zhikang T. Wang Paul Pu Liang Ruslan Salakhutdinov Louis-Philippe Morency Masahito Ueda 23 110 0 29 Jun 2019
Shallow Neural Networks for Fluid Flow Reconstruction with Limited Sensors N. Benjamin Erichson L. Mathelin Z. Yao Steven L. Brunton Michael W. Mahoney J. Nathan Kutz AI4CE 27 34 0 20 Feb 2019
Parameter Efficient Training of Deep Convolutional Neural Networks by Dynamic Sparse Reparameterization Hesham Mostafa Xin Wang 37 307 0 15 Feb 2019
Optimization Problems for Machine Learning: A Survey Claudio Gambella Bissan Ghaddar Joe Naoum-Sawaya AI4CE 30 178 0 16 Jan 2019
Towards Theoretical Understanding of Large Batch Training in Stochastic Gradient Descent Xiaowu Dai Yuhua Zhu 25 11 0 03 Dec 2018
Generalization Error in Deep Learning Daniel Jakubovitz Raja Giryes M. Rodrigues AI4CE 32 109 0 03 Aug 2018
Data augmentation instead of explicit regularization Alex Hernández-García Peter König 30 141 0 11 Jun 2018
Optimal ridge penalty for real-world high-dimensional data can be zero or negative due to the implicit ridge regularization D. Kobak Jonathan Lomond Benoit Sanchez 33 89 0 28 May 2018
High Dimensional Spaces, Deep Learning and Adversarial Examples S. Dube 37 29 0 02 Jan 2018
Three Factors Influencing Minima in SGD Stanislaw Jastrzebski Zachary Kenton Devansh Arpit Nicolas Ballas Asja Fischer Yoshua Bengio Amos Storkey 42 457 0 13 Nov 2017