Grokking as a First Order Phase Transition in Two Layer Networks

5 October 2023

Papers citing "Grokking as a First Order Phase Transition in Two Layer Networks"

14 / 14 papers shown

Title
Position: Solve Layerwise Linear Models First to Understand Neural Dynamical Phenomena (Neural Collapse, Emergence, Lazy/Rich Regime, and Grokking) Yoonsoo Nam Seok Hyeong Lee Clementine Domine Yea Chan Park Charles London Wonyl Choi Niclas Goring Seungjai Lee AI4CE 33 0 0 28 Feb 2025
Grokking at the Edge of Numerical Stability Lucas Prieto Melih Barsbey Pedro A.M. Mediano Tolga Birdal 32 3 0 08 Jan 2025
Grokking at the Edge of Linear Separability Alon Beck Noam Levi Yohai Bar-Sinai 29 0 0 06 Oct 2024
Why Do You Grok? A Theoretical Analysis of Grokking Modular Addition Mohamad Amin Mohamadi Zhiyuan Li Lei Wu Danica J. Sutherland 40 10 0 17 Jul 2024
Get rich quick: exact solutions reveal how unbalanced initializations promote rapid feature learning D. Kunin Allan Raventós Clémentine Dominé Feng Chen David Klindt Andrew M. Saxe Surya Ganguli MLT 40 15 0 10 Jun 2024
Grokking Modular Polynomials Darshil Doshi Tianyu He Aritra Das Andrey Gromov 40 4 0 05 Jun 2024
Phase Transitions in the Output Distribution of Large Language Models Julian Arnold Flemming Holtorf Frank Schafer Niels Lörch 39 1 0 27 May 2024
Bridging Lottery Ticket and Grokking: Understanding Grokking from Inner Structure of Networks Gouki Minegishi Yusuke Iwasawa Yutaka Matsuo 11 3 0 30 Oct 2023
Spectral-Bias and Kernel-Task Alignment in Physically Informed Neural Networks Inbar Seroussi Asaf Miron Z. Ringel PINN 32 0 0 12 Jul 2023
Learning Single-Index Models with Shallow Neural Networks A. Bietti Joan Bruna Clayton Sanford M. Song 162 67 0 27 Oct 2022
Grokking phase transitions in learning local rules with gradient descent Bojan Žunkovič E. Ilievski 55 16 0 26 Oct 2022
Omnigrok: Grokking Beyond Algorithmic Data Ziming Liu Eric J. Michaud Max Tegmark 56 76 0 03 Oct 2022
Neural Networks Efficiently Learn Low-Dimensional Representations with SGD Alireza Mousavi-Hosseini Sejun Park M. Girotti Ioannis Mitliagkas Murat A. Erdogdu MLT 319 48 0 29 Sep 2022
MCMC using Hamiltonian dynamics Radford M. Neal 173 3,260 0 09 Jun 2012