Residual Networks as Nonlinear Systems: Stability Analysis using Linearization

31 May 2019

Papers citing "Residual Networks as Nonlinear Systems: Stability Analysis using Linearization"

9 / 9 papers shown

Title
Dynamical Isometry is Achieved in Residual Networks in a Universal Way for any Activation Function W. Tarnowski P. Warchol Stanislaw Jastrzebski Jacek Tabor M. Nowak 40 38 0 24 Sep 2018
Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks Lechao Xiao Yasaman Bahri Jascha Narain Sohl-Dickstein S. Schoenholz Jeffrey Pennington 298 353 0 14 Jun 2018
The Emergence of Spectral Universality in Deep Networks Jeffrey Pennington S. Schoenholz Surya Ganguli 56 173 0 27 Feb 2018
Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice Jeffrey Pennington S. Schoenholz Surya Ganguli ODL 43 252 0 13 Nov 2017
Adversarial examples in the physical world Alexey Kurakin Ian Goodfellow Samy Bengio SILM AAML 517 5,885 0 08 Jul 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 1.8K 193,426 0 10 Dec 2015
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift Sergey Ioffe Christian Szegedy OOD 409 43,234 0 11 Feb 2015
Explaining and Harnessing Adversarial Examples Ian Goodfellow Jonathon Shlens Christian Szegedy AAML GAN 225 19,017 0 20 Dec 2014
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks Andrew M. Saxe James L. McClelland Surya Ganguli ODL 162 1,844 0 20 Dec 2013