Entropic alternatives to initialization

16 July 2021

Papers citing "Entropic alternatives to initialization"

15 / 15 papers shown

Title
Towards quantifying information flows: relative entropy in deep neural networks and the renormalization group J. Erdmenger Kevin T. Grosvenor R. Jefferson 59 17 0 14 Jul 2021
Partial local entropy and anisotropy in deep weight spaces Daniele Musso 11 3 0 17 Jul 2020
A scale-dependent notion of effective dimension Oksana Berezniuk Alessio Figalli Raffaele Ghigliazza Kharen Musaelian 25 24 0 29 Jan 2020
Mean-field inference methods for neural networks Marylou Gabrié AI4CE 41 33 0 03 Nov 2019
Properties of the geometry of solutions and capacity of multi-layer neural networks with Rectified Linear Units activations Carlo Baldassi Enrico M. Malatesta R. Zecchina MLT 6 43 0 17 Jul 2019
Shaping the learning landscape in neural networks around wide flat minima Carlo Baldassi Fabrizio Pittorino R. Zecchina MLT 30 82 0 20 May 2019
Improved Regularization of Convolutional Neural Networks with Cutout Terrance Devries Graham W. Taylor 63 3,739 0 15 Aug 2017
Opening the Black Box of Deep Neural Networks via Information Ravid Shwartz-Ziv Naftali Tishby AI4CE 61 1,399 0 02 Mar 2017
Deep Information Propagation S. Schoenholz Justin Gilmer Surya Ganguli Jascha Narain Sohl-Dickstein 39 363 0 04 Nov 2016
Unreasonable Effectiveness of Learning Neural Networks: From Accessible States and Robust Ensembles to Basic Algorithmic Schemes Carlo Baldassi C. Borgs J. Chayes Alessandro Ingrosso Carlo Lucibello Luca Saglietti R. Zecchina 40 166 0 20 May 2016
Local entropy as a measure for sampling solutions in Constraint Satisfaction Problems Carlo Baldassi Alessandro Ingrosso Carlo Lucibello Luca Saglietti R. Zecchina 29 57 0 18 Nov 2015
Subdominant Dense Clusters Allow for Simple Learning and High Computational Performance in Neural Networks with Discrete Synapses Carlo Baldassi Alessandro Ingrosso Carlo Lucibello Luca Saglietti R. Zecchina 32 127 0 18 Sep 2015
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification Kaiming He Xinming Zhang Shaoqing Ren Jian Sun VLM 95 18,534 0 06 Feb 2015
Deep learning with Elastic Averaging SGD Sixin Zhang A. Choromańska Yann LeCun FedML 38 609 0 20 Dec 2014
Revisiting Natural Gradient for Deep Networks Razvan Pascanu Yoshua Bengio ODL 78 388 0 16 Jan 2013