Entropy and mutual information in models of deep neural networks

Entropy and mutual information in models of deep neural networks

24 May 2018

Marylou Gabrié

Clément Luneau

Florent Krzakala

Lenka Zdeborová

Papers citing "Entropy and mutual information in models of deep neural networks"

10 / 10 papers shown

Title
High-entropy Advantage in Neural Networks' Generalizability Entao Yang Wei Wei Yue Shang Ge Zhang AI4CE 80 0 0 17 Mar 2025
Connecting NTK and NNGP: A Unified Theoretical Framework for Wide Neural Network Learning Dynamics Yehonatan Avidan Qianyi Li H. Sompolinsky 83 8 0 08 Sep 2023
Conditional Mutual Information Neural Estimator Sina Molavipour Germán Bassi Mikael Skoglund 48 14 0 06 Nov 2019
Optimal Errors and Phase Transitions in High-Dimensional Generalized Linear Models Jean Barbier Florent Krzakala N. Macris Léo Miolane Lenka Zdeborová 62 262 0 10 Aug 2017
ACDC: A Structured Efficient Linear Layer Marcin Moczulski Misha Denil J. Appleyard Nando de Freitas 59 98 0 18 Nov 2015
Deep Neural Networks with Random Gaussian Weights: A Universal Classification Strategy? Raja Giryes Guillermo Sapiro A. Bronstein 75 187 0 30 Apr 2015
Deep Learning and the Information Bottleneck Principle Naftali Tishby Noga Zaslavsky DRL 123 1,570 0 09 Mar 2015
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks Andrew M. Saxe James L. McClelland Surya Ganguli ODL 114 1,830 0 20 Dec 2013
High Dimensional Robust M-Estimation: Asymptotic Variance via Approximate Message Passing D. Donoho Andrea Montanari 144 221 0 28 Oct 2013
Efficient supervised learning in networks with binary synapses Carlo Baldassi Alfredo Braunstein Nicolas Brunel R. Zecchina 176 110 0 09 Jul 2007