Network size and weights size for memorization with two-layers neural
networks

Network size and weights size for memorization with two-layers neural networks

4 June 2020

Sébastien Bubeck

Papers citing "Network size and weights size for memorization with two-layers neural networks"

11 / 11 papers shown

Title
Analysis of the expected $L_2$ error of an over-parametrized deep neural network estimate learned by gradient descent without regularization Selina Drews Michael Kohler 36 2 0 24 Nov 2023
Memorization Capacity of Multi-Head Attention in Transformers Sadegh Mahdavi Renjie Liao Christos Thrampoulidis 26 22 0 03 Jun 2023
When Expressivity Meets Trainability: Fewer than $n$ Neurons Can Work Jiawei Zhang Yushun Zhang Mingyi Hong Ruoyu Sun Zhi-Quan Luo 26 10 0 21 Oct 2022
Why Robust Generalization in Deep Learning is Difficult: Perspective of Expressive Power Binghui Li Jikai Jin Han Zhong J. Hopcroft Liwei Wang OOD 82 27 0 27 May 2022
NeuFENet: Neural Finite Element Solutions with Theoretical Bounds for Parametric PDEs Biswajit Khara Aditya Balu Ameya Joshi S. Sarkar C. Hegde A. Krishnamurthy Baskar Ganapathysubramanian 26 19 0 04 Oct 2021
Knowledge Infused Policy Gradients for Adaptive Pandemic Control Kaushik Roy Qi Zhang Manas Gaur A. Sheth 19 12 0 11 Feb 2021
Tight Bounds on the Smallest Eigenvalue of the Neural Tangent Kernel for Deep ReLU Networks Quynh N. Nguyen Marco Mondelli Guido Montúfar 25 81 0 21 Dec 2020
A law of robustness for two-layers neural networks Sébastien Bubeck Yuanzhi Li Dheeraj M. Nagaraj 33 57 0 30 Sep 2020
The Interpolation Phase Transition in Neural Networks: Memorization and Generalization under Lazy Training Andrea Montanari Yiqiao Zhong 47 95 0 25 Jul 2020
Training (Overparametrized) Neural Networks in Near-Linear Time Jan van den Brand Binghui Peng Zhao Song Omri Weinstein ODL 26 82 0 20 Jun 2020
Norm-Based Capacity Control in Neural Networks Behnam Neyshabur Ryota Tomioka Nathan Srebro 127 577 0 27 Feb 2015