On the Learning Dynamics of Deep Neural Networks

On the Learning Dynamics of Deep Neural Networks

18 September 2018

Rémi Tachet des Combes

Mohammad Pezeshki

Samira Shabanian

Aaron Courville

Papers citing "On the Learning Dynamics of Deep Neural Networks"

9 / 9 papers shown

Title
Learned feature representations are biased by complexity, learning order, position, and more Andrew Kyle Lampinen Stephanie C. Y. Chan Katherine Hermann AI4CE FaML SSL OOD 34 6 0 09 May 2024
Neural Redshift: Random Networks are not Random Functions Damien Teney A. Nicolicioiu Valentin Hartmann Ehsan Abbasnejad 97 18 0 04 Mar 2024
The Neural Race Reduction: Dynamics of Abstraction in Gated Networks Andrew M. Saxe Shagun Sodhani Sam Lewallen AI4CE 28 34 0 21 Jul 2022
Towards Understanding the Data Dependency of Mixup-style Training Muthuraman Chidambaram Xiang Wang Yuzheng Hu Chenwei Wu Rong Ge UQCV 47 24 0 14 Oct 2021
Gradient Starvation: A Learning Proclivity in Neural Networks Mohammad Pezeshki Sekouba Kaba Yoshua Bengio Aaron Courville Doina Precup Guillaume Lajoie MLT 45 257 0 18 Nov 2020
An analytic theory of shallow networks dynamics for hinge loss classification Franco Pellegrini Giulio Biroli 29 19 0 19 Jun 2020
Unsupervised State Representation Learning in Atari Ankesh Anand Evan Racah Sherjil Ozair Yoshua Bengio Marc-Alexandre Côté R. Devon Hjelm SSL 27 253 0 19 Jun 2019
An Empirical Study of Example Forgetting during Deep Neural Network Learning Mariya Toneva Alessandro Sordoni Rémi Tachet des Combes Adam Trischler Yoshua Bengio Geoffrey J. Gordon 46 712 0 12 Dec 2018
When Will Gradient Methods Converge to Max-margin Classifier under ReLU Models? Tengyu Xu Yi Zhou Kaiyi Ji Yingbin Liang 21 19 0 12 Jun 2018