Deconstructing the Goldilocks Zone of Neural Network Initialization

5 February 2024

Papers citing "Deconstructing the Goldilocks Zone of Neural Network Initialization"

4 / 4 papers shown

Title
Information-Theoretic Progress Measures reveal Grokking is an Emergent Phase Transition Kenzo Clauw S. Stramaglia Daniele Marinazzo 50 3 0 16 Aug 2024
Mitigating Neural Network Overconfidence with Logit Normalization Hongxin Wei Renchunzi Xie Hao-Ran Cheng Lei Feng Bo An Yixuan Li OODD 163 267 0 19 May 2022
The large learning rate phase of deep learning: the catapult mechanism Aitor Lewkowycz Yasaman Bahri Ethan Dyer Jascha Narain Sohl-Dickstein Guy Gur-Ari ODL 159 235 0 04 Mar 2020
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 310 2,892 0 15 Sep 2016