Echo State Condition at the Critical Point

Recurrent networks that have transfer functions that fulfill the Lipschitz continuity with $L=1$, may be echo state networks if certain limitations on the recurrent connectivity are applied. Initially it has been shown that it is sufficient if the largest singular value of the recurrent connectivity $S$ is smaller than 1. The main achievement of this paper is a proof under which conditions the network is an echo state network even if $S=1$. It turns out that in this critical case the exact shape of the transfer function plays a decisive role whether or not the network still fulfills the echo state condition. In addition, several intuitive examples with one neuron networks are outlined to illustrate how a critical connectivity relate to a certain interesting behavior to a certain, anticipated input statistics.