Wide Neural Networks Trained with Weight Decay Provably Exhibit Neural
Collapse

Wide Neural Networks Trained with Weight Decay Provably Exhibit Neural Collapse

7 October 2024

Peter Súkeník

ArXiv (abs)PDF HTML

Papers citing "Wide Neural Networks Trained with Weight Decay Provably Exhibit Neural Collapse"

17 / 17 papers shown

Title
Average gradient outer product as a mechanism for deep neural collapse Daniel Beaglehole Peter Súkeník Marco Mondelli Misha Belkin AI4CE 67 13 0 21 Feb 2024
Which Frequencies do CNNs Need? Emergent Bottleneck Structure in Feature Learning Yuxiao Wen Arthur Jacot 105 7 0 12 Feb 2024
Neural Collapse for Unconstrained Feature Model under Cross-entropy Loss with Imbalanced Data Wanli Hong Shuyang Ling 53 17 0 18 Sep 2023
Towards Understanding Neural Collapse: The Effects of Batch Normalization and Weight Decay Leyan Pan Xinyuan Cao 40 5 0 09 Sep 2023
No Fear of Classifier Biases: Neural Collapse Inspired Federated Learning with Synthetic and Fixed Classifier Zexi Li Xinyi Shang Rui He Tao R. Lin Chao Wu FedML 91 55 0 17 Mar 2023
Learning Single-Index Models with Shallow Neural Networks A. Bietti Joan Bruna Clayton Sanford M. Song 188 71 0 27 Oct 2022
Are All Losses Created Equal: A Neural Collapse Perspective Jinxin Zhou Chong You Xiao Li Kangning Liu Sheng Liu Qing Qu Zhihui Zhu 72 64 0 04 Oct 2022
Linking Neural Collapse and L2 Normalization with Improved Out-of-Distribution Detection in Deep Neural Networks J. Haas William Yolland B. Rabus OODD 66 19 0 17 Sep 2022
Imbalance Trouble: Revisiting Neural-Collapse Geometry Christos Thrampoulidis Ganesh Ramachandra Kini V. Vakilian Tina Behnia 58 73 0 10 Aug 2022
$Feature Learning in $L_{2}$-regularized DNNs: Attraction/Repulsion and Sparsity$ Feature Learning in $L_{2}$ -regularized DNNs: Attraction/Repulsion and Sparsity Arthur Jacot Eugene Golikov Clément Hongler Franck Gabriel MLT 63 18 0 31 May 2022
Memorization and Optimization in Deep Neural Networks with Minimum Over-parameterization Simone Bombari Mohammad Hossein Amani Marco Mondelli 64 26 0 20 May 2022
An Unconstrained Layer-Peeled Perspective on Neural Collapse Wenlong Ji Yiping Lu Yiliang Zhang Zhun Deng Weijie J. Su 176 86 0 06 Oct 2021
Neural collapse with unconstrained features D. Mixon Hans Parshall Jianzong Pi 72 120 0 23 Nov 2020
Prevalence of Neural Collapse during the terminal phase of deep learning training Vardan Papyan Xuemei Han D. Donoho 202 578 0 18 Aug 2020
The Barron Space and the Flow-induced Function Spaces for Neural Network Models E. Weinan Chao Ma Lei Wu 71 110 0 18 Jun 2019
Implicit Regularization in Deep Matrix Factorization Sanjeev Arora Nadav Cohen Wei Hu Yuping Luo AI4CE 85 504 0 31 May 2019
On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization Sanjeev Arora Nadav Cohen Elad Hazan 102 485 0 19 Feb 2018