Deep Double Descent: Where Bigger Models and More Data Hurt

4 December 2019

Papers citing "Deep Double Descent: Where Bigger Models and More Data Hurt"

50 / 205 papers shown

Title
On Memorization in Probabilistic Deep Generative Models G. V. D. Burg Christopher K. I. Williams TDI 25 59 0 06 Jun 2021
Post-mortem on a deep learning contest: a Simpson's paradox and the complementary roles of scale metrics versus shape metrics Charles H. Martin Michael W. Mahoney 18 19 0 01 Jun 2021
A Universal Law of Robustness via Isoperimetry Sébastien Bubeck Mark Sellke 13 213 0 26 May 2021
Receptive Field Regularization Techniques for Audio Classification and Tagging with Deep Convolutional Neural Networks Khaled Koutini Hamid Eghbalzadeh Gerhard Widmer 30 46 0 26 May 2021
A brain basis of dynamical intelligence for AI and computational neuroscience J. Monaco Kanaka Rajan Grace M. Hwang AI4CE 26 6 0 15 May 2021
A Geometric Analysis of Neural Collapse with Unconstrained Features Zhihui Zhu Tianyu Ding Jinxin Zhou Xiao Li Chong You Jeremias Sulam Qing Qu 27 194 0 06 May 2021
Deep limits and cut-off phenomena for neural networks B. Avelin A. Karlsson AI4CE 38 2 0 21 Apr 2021
Low-Regret Active learning Cenk Baykal Lucas Liebenwein Dan Feldman Daniela Rus UQCV 36 3 0 06 Apr 2021
The Shape of Learning Curves: a Review T. Viering Marco Loog 18 122 0 19 Mar 2021
Lost in Pruning: The Effects of Pruning Neural Networks beyond Test Accuracy Lucas Liebenwein Cenk Baykal Brandon Carter David K Gifford Daniela Rus AAML 40 71 0 04 Mar 2021
Label-Imbalanced and Group-Sensitive Classification under Overparameterization Ganesh Ramachandra Kini Orestis Paraskevas Samet Oymak Christos Thrampoulidis 27 93 0 02 Mar 2021
GIST: Distributed Training for Large-Scale Graph Convolutional Networks Cameron R. Wolfe Jingkang Yang Arindam Chowdhury Chen Dun Artun Bayer Santiago Segarra Anastasios Kyrillidis BDL GNN LRM 51 9 0 20 Feb 2021
Low Curvature Activations Reduce Overfitting in Adversarial Training Vasu Singla Sahil Singla David Jacobs S. Feizi AAML 32 45 0 15 Feb 2021
Explaining Neural Scaling Laws Yasaman Bahri Ethan Dyer Jared Kaplan Jaehoon Lee Utkarsh Sharma 27 250 0 12 Feb 2021
DICE: Diversity in Deep Ensembles via Conditional Redundancy Adversarial Estimation Alexandre Ramé Matthieu Cord FedML 53 51 0 14 Jan 2021
AutoDropout: Learning Dropout Patterns to Regularize Deep Networks Hieu H. Pham Quoc V. Le 76 56 0 05 Jan 2021
Analysis of the Scalability of a Deep-Learning Network for Steganography "Into the Wild" Hugo Ruiz Marc Chaumont Mehdi Yedroudj A. Amara Frédéric Comby Gérard Subsol 29 9 0 29 Dec 2020
Applying Deutsch's concept of good explanations to artificial intelligence and neuroscience -- an initial exploration Daniel C. Elton 23 4 0 16 Dec 2020
Provable Benefits of Overparameterization in Model Compression: From Double Descent to Pruning Neural Networks Xiangyu Chang Yingcong Li Samet Oymak Christos Thrampoulidis 35 50 0 16 Dec 2020
A case for new neural network smoothness constraints Mihaela Rosca T. Weber Arthur Gretton S. Mohamed AAML 33 48 0 14 Dec 2020
Underspecification Presents Challenges for Credibility in Modern Machine Learning Alexander DÁmour Katherine A. Heller D. Moldovan Ben Adlam B. Alipanahi ... Kellie Webster Steve Yadlowsky T. Yun Xiaohua Zhai D. Sculley OffRL 77 670 0 06 Nov 2020
Understanding Double Descent Requires a Fine-Grained Bias-Variance Decomposition Ben Adlam Jeffrey Pennington UD 39 93 0 04 Nov 2020
A Bayesian Perspective on Training Speed and Model Selection Clare Lyle Lisa Schut Binxin Ru Y. Gal Mark van der Wilk 44 24 0 27 Oct 2020
Memorizing without overfitting: Bias, variance, and interpolation in over-parameterized models J. Rocks Pankaj Mehta 23 41 0 26 Oct 2020
State space models for building control: how deep should you go? B. Schubnel R. Carrillo Paolo Taddeo L. C. Casals J. Salom Y. Stauffer P. Alet 27 14 0 23 Oct 2020
Data-Efficient Pretraining via Contrastive Self-Supervision Nils Rethmeier Isabelle Augenstein 23 20 0 02 Oct 2020
Effective Regularization Through Loss-Function Metalearning Santiago Gonzalez Risto Miikkulainen 29 5 0 02 Oct 2020
Small Data, Big Decisions: Model Selection in the Small-Data Regime J. Bornschein Francesco Visin Simon Osindero 15 36 0 26 Sep 2020
Efficient Quantum State Sample Tomography with Basis-dependent Neural-networks Alistair W. R. Smith Johnnie Gray M. S. Kim 11 28 0 16 Sep 2020
Statistical Query Algorithms and Low-Degree Tests Are Almost Equivalent Matthew Brennan Guy Bresler Samuel B. Hopkins Jingkai Li T. Schramm 19 62 0 13 Sep 2020
Unsupervised MRI Reconstruction with Generative Adversarial Networks Elizabeth K. Cole John M. Pauly S. Vasanawala Frank Ong GAN MedIm 19 50 0 29 Aug 2020
Multiple Descent: Design Your Own Generalization Curve Lin Chen Yifei Min M. Belkin Amin Karbasi DRL 28 61 0 03 Aug 2020
Exploring Weight Importance and Hessian Bias in Model Pruning Mingchen Li Yahya Sattar Christos Thrampoulidis Samet Oymak 28 3 0 19 Jun 2020
When Does Preconditioning Help or Hurt Generalization? S. Amari Jimmy Ba Roger C. Grosse Xuechen Li Atsushi Nitanda Taiji Suzuki Denny Wu Ji Xu 36 32 0 18 Jun 2020
To Pretrain or Not to Pretrain: Examining the Benefits of Pretraining on Resource Rich Tasks Sinong Wang Madian Khabsa Hao Ma 18 26 0 15 Jun 2020
Double Descent Risk and Volume Saturation Effects: A Geometric Perspective Prasad Cheema M. Sugiyama 14 3 0 08 Jun 2020
An Overview of Neural Network Compression James OÑeill AI4CE 45 98 0 05 Jun 2020
An Investigation of Why Overparameterization Exacerbates Spurious Correlations Shiori Sagawa Aditi Raghunathan Pang Wei Koh Percy Liang 152 371 0 09 May 2020
Random Features for Kernel Approximation: A Survey on Algorithms, Theory, and Beyond Fanghui Liu Xiaolin Huang Yudong Chen Johan A. K. Suykens BDL 44 172 0 23 Apr 2020
Going in circles is the way forward: the role of recurrence in visual inference R. S. V. Bergen N. Kriegeskorte 17 82 0 26 Mar 2020
Double Trouble in Double Descent : Bias and Variance(s) in the Lazy Regime Stéphane dÁscoli Maria Refinetti Giulio Biroli Florent Krzakala 93 152 0 02 Mar 2020
Overfitting in adversarially robust deep learning Leslie Rice Eric Wong Zico Kolter 47 785 0 26 Feb 2020
The Curious Case of Adversarially Robust Models: More Data Can Help, Double Descend, or Hurt Generalization Yifei Min Lin Chen Amin Karbasi AAML 37 69 0 25 Feb 2020
Generalisation error in learning with random features and the hidden manifold model Federica Gerace Bruno Loureiro Florent Krzakala M. Mézard Lenka Zdeborová 25 165 0 21 Feb 2020
Bayesian Deep Learning and a Probabilistic Perspective of Generalization A. Wilson Pavel Izmailov UQCV BDL OOD 24 639 0 20 Feb 2020
Implicit Regularization of Random Feature Models Arthur Jacot Berfin Simsek Francesco Spadaro Clément Hongler Franck Gabriel 31 82 0 19 Feb 2020
A Model of Double Descent for High-dimensional Binary Linear Classification Zeyu Deng A. Kammoun Christos Thrampoulidis 36 145 0 13 Nov 2019
FLAML: A Fast and Lightweight AutoML Library Chi Wang Qingyun Wu Markus Weimer Erkang Zhu 30 196 0 12 Nov 2019
Capacity, Bandwidth, and Compositionality in Emergent Language Learning Cinjon Resnick Abhinav Gupta Jakob N. Foerster Andrew M. Dai Kyunghyun Cho 20 51 0 24 Oct 2019
Predicting materials properties without crystal structure: Deep representation learning from stoichiometry Rhys E. A. Goodall A. Lee 15 253 0 01 Oct 2019