Information Geometry of Orthogonal Initializations and Training

9 October 2018

Papers citing "Information Geometry of Orthogonal Initializations and Training"

4 / 4 papers shown

Title
Asymptotic Freeness of Layerwise Jacobians Caused by Invariance of Multilayer Perceptron: The Haar Orthogonal Case B. Collins Tomohiro Hayase 22 7 0 24 Mar 2021
The Spectrum of Fisher Information of Deep Networks Achieving Dynamical Isometry Tomohiro Hayase Ryo Karakida 29 7 0 14 Jun 2020
Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks Lechao Xiao Yasaman Bahri Jascha Narain Sohl-Dickstein S. Schoenholz Jeffrey Pennington 244 348 0 14 Jun 2018
Optimization on Submanifolds of Convolution Kernels in CNNs Mete Ozay Takayuki Okatani 51 46 0 22 Oct 2016