Average gradient outer product as a mechanism for deep neural collapse

Average gradient outer product as a mechanism for deep neural collapse

21 February 2024

Daniel Beaglehole

Peter Súkeník

Papers citing "Average gradient outer product as a mechanism for deep neural collapse"

17 / 17 papers shown

Title
Feature maps for the Laplacian kernel and its generalizations Sudhendu Ahir Parthe Pandit 60 0 0 24 Feb 2025
Theoretical characterisation of the Gauss-Newton conditioning in Neural Networks Jim Zhao Sidak Pal Singh Aurelien Lucchi AI4CE 45 0 0 04 Nov 2024
The Persistence of Neural Collapse Despite Low-Rank Bias: An Analytic Perspective Through Unconstrained Features Connall Garrod Jonathan P. Keating 36 2 0 30 Oct 2024
Wide Neural Networks Trained with Weight Decay Provably Exhibit Neural Collapse Arthur Jacot Peter Súkeník Zihan Wang Marco Mondelli 31 1 0 07 Oct 2024
Emergence in non-neural models: grokking modular arithmetic via average gradient outer product Neil Rohit Mallinar Daniel Beaglehole Libin Zhu Adityanarayanan Radhakrishnan Parthe Pandit Misha Belkin 51 7 0 29 Jul 2024
Compressible Dynamics in Deep Overparameterized Low-Rank Learning & Adaptation Can Yaras Peng Wang Laura Balzano Qing Qu AI4CE 37 12 0 06 Jun 2024
Linguistic Collapse: Neural Collapse in (Large) Language Models Robert Wu V. Papyan 48 12 0 28 May 2024
Neural Collapse versus Low-rank Bias: Is Deep Neural Collapse Really Optimal? Peter Súkeník Marco Mondelli Christoph H. Lampert AI4CE 34 7 0 23 May 2024
The fast committor machine: Interpretable prediction with kernels D. Aristoff Mats S. Johnson Gideon Simpson Robert J. Webber 26 5 0 16 May 2024
Feature learning as alignment: a structural property of gradient descent in non-linear neural networks Daniel Beaglehole Ioannis Mitliagkas Atish Agarwala MLT 34 2 0 07 Feb 2024
Linear Recursive Feature Machines provably recover low-rank matrices Adityanarayanan Radhakrishnan Misha Belkin Dmitriy Drusvyatskiy 58 8 0 09 Jan 2024
ReLU Neural Networks with Linear Layers are Biased Towards Single- and Multi-Index Models Suzanna Parkinson Greg Ongie Rebecca Willett 62 6 0 24 May 2023
Neural Collapse in Deep Linear Networks: From Balanced to Imbalanced Data Hien Dang Tho Tran T. Nguyen Hung The Tran Nhat Ho Hung Tran 40 28 0 01 Jan 2023
Perturbation Analysis of Neural Collapse Tom Tirer Haoxiang Huang Jonathan Niles-Weed AAML 35 23 0 29 Oct 2022
Linking Neural Collapse and L2 Normalization with Improved Out-of-Distribution Detection in Deep Neural Networks J. Haas William Yolland B. Rabus OODD 41 14 0 17 Sep 2022
An Unconstrained Layer-Peeled Perspective on Neural Collapse Wenlong Ji Yiping Lu Yiliang Zhang Zhun Deng Weijie J. Su 132 83 0 06 Oct 2021
Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Training Cong Fang Hangfeng He Qi Long Weijie J. Su FAtt 130 165 0 29 Jan 2021