On overcoming the Curse of Dimensionality in Neural Networks
Let be a reproducing Kernel Hilbert space. For , let and comprise our dataset. Let be the unique global minimiser of the functional \begin{equation*} J(f) = \frac{1}{2}\Vert f\Vert_{H}^{2} + \frac{1}{N}\sum_{i=1}^{N}\frac{1}{2}\vert f(x_i)-y_i\vert^{2}. \end{equation*} In this paper we show that for each there exists a two layer network where the first layer has number of basis functions for , and the second layer takes a weighted summation of the first layer, such that the functions realised by these networks satisfy \begin{equation*} \Vert f_{n}-f^*\Vert_{H}\leq O(\frac{1}{\sqrt{n}})\enspace \text{for all}\enspace n\in\mathbb{N}. \end{equation*} Thus the error rate is independent of input dimension , output dimension and data size .
View on arXiv