Statistically guided deep learning

We present a theoretically well-founded deep learning algorithm for nonparametric regression. It uses over-parametrized deep neural networks with logistic activation function, which are fitted to the given data via gradient descent. We propose a special topology of these networks, a special random initialization of the weights, and a data-dependent choice of the learning rate and the number of gradient descent steps. We prove a theoretical bound on the expected $L_2$ error of this estimate, and illustrate its finite sample size performance by applying it to simulated data. Our results show that a theoretical analysis of deep learning which takes into account simultaneously optimization, generalization and approximation can result in a new deep learning estimate which has an improved finite sample performance.

@article{kohler2025_2504.08489,
  title={ Statistically guided deep learning },
  author={ Michael Kohler and Adam Krzyzak },
  journal={arXiv preprint arXiv:2504.08489},
  year={ 2025 }
}