Entropy and mutual information in models of deep neural networks

Entropy and mutual information in models of deep neural networks

24 May 2018

Marylou Gabrié

Clément Luneau

Florent Krzakala

Lenka Zdeborová

Papers citing "Entropy and mutual information in models of deep neural networks"

9 / 9 papers shown

Title
High-entropy Advantage in Neural Networks' Generalizability Entao Yang Wei Wei Yue Shang Ge Zhang AI4CE 75 0 0 17 Mar 2025
Connecting NTK and NNGP: A Unified Theoretical Framework for Wide Neural Network Learning Dynamics Yehonatan Avidan Qianyi Li H. Sompolinsky 83 8 0 08 Sep 2023
Conditional Mutual Information Neural Estimator Sina Molavipour Germán Bassi Mikael Skoglund 43 14 0 06 Nov 2019
Optimal Errors and Phase Transitions in High-Dimensional Generalized Linear Models Jean Barbier Florent Krzakala N. Macris Léo Miolane Lenka Zdeborová 60 262 0 10 Aug 2017
ACDC: A Structured Efficient Linear Layer Marcin Moczulski Misha Denil J. Appleyard Nando de Freitas 54 98 0 18 Nov 2015
Deep Neural Networks with Random Gaussian Weights: A Universal Classification Strategy? Raja Giryes Guillermo Sapiro A. Bronstein 65 187 0 30 Apr 2015
Deep Learning and the Information Bottleneck Principle Naftali Tishby Noga Zaslavsky DRL 116 1,570 0 09 Mar 2015
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks Andrew M. Saxe James L. McClelland Surya Ganguli ODL 105 1,830 0 20 Dec 2013
Efficient supervised learning in networks with binary synapses Carlo Baldassi Alfredo Braunstein Nicolas Brunel R. Zecchina 147 110 0 09 Jul 2007