A Geometric Analysis of Neural Collapse with Unconstrained Features

6 May 2021

Qing Qu

Papers citing "A Geometric Analysis of Neural Collapse with Unconstrained Features"

48 / 148 papers shown

Title
Factorizers for Distributed Sparse Block Codes Michael Hersche Aleksandar Terzić G. Karunaratne Jovin Langenegger Angeline Pouget G. Cherubini Luca Benini Abu Sebastian Abbas Rahimi 39 4 0 24 Mar 2023
No Fear of Classifier Biases: Neural Collapse Inspired Federated Learning with Synthetic and Fixed Classifier Zexi Li Xinyi Shang Rui He Tao R. Lin Chao Wu FedML 51 54 0 17 Mar 2023
On the Implicit Geometry of Cross-Entropy Parameterizations for Label-Imbalanced Data Tina Behnia Ganesh Ramachandra Kini V. Vakilian Christos Thrampoulidis 44 17 0 14 Mar 2023
Generalizing and Decoupling Neural Collapse via Hyperspherical Uniformity Gap Weiyang Liu L. Yu Adrian Weller Bernhard Schölkopf 37 18 0 11 Mar 2023
A Message Passing Perspective on Learning Dynamics of Contrastive Learning Yifei Wang Qi Zhang Tianqi Du Jiansheng Yang Zhouchen Lin Yisen Wang SSL 32 18 0 08 Mar 2023
Inducing Neural Collapse in Deep Long-tailed Learning Xuantong Liu Jianfeng Zhang Tianyang Hu He Cao Lujia Pan Yuan Yao 27 27 0 24 Feb 2023
Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class Incremental Learning Yibo Yang Haobo Yuan Hefei Ling Zhouchen Lin Philip Torr Dacheng Tao CLL 34 98 0 06 Feb 2023
Unsupervised Manifold Linearizing and Clustering Tianjiao Ding Shengbang Tong Kwan Ho Ryan Chan Xili Dai Yi Ma B. Haeffele 25 11 0 04 Jan 2023
Understanding Imbalanced Semantic Segmentation Through Neural Collapse Zhisheng Zhong Jiequan Cui Yibo Yang Xiaoyang Wu Xiaojuan Qi Xinming Zhang Jiaya Jia 138 45 0 03 Jan 2023
Pruning Before Training May Improve Generalization, Provably Hongru Yang Yingbin Liang Xiaojie Guo Lingfei Wu Zhangyang Wang MLT 24 1 0 01 Jan 2023
Generalization Bounds for Few-Shot Transfer Learning with Pretrained Classifiers Tomer Galanti András Gyorgy Marcus Hutter VLM SSL 26 3 0 23 Dec 2022
Principled and Efficient Transfer Learning of Deep Models via Neural Collapse Xiao Li Sheng Liu Jin-li Zhou Xin Lu C. Fernandez‐Granda Zhihui Zhu Q. Qu AAML 28 19 0 23 Dec 2022
Perturbation Analysis of Neural Collapse Tom Tirer Haoxiang Huang Jonathan Niles-Weed AAML 48 24 0 29 Oct 2022
Grokking phase transitions in learning local rules with gradient descent Bojan Žunkovič E. Ilievski 63 16 0 26 Oct 2022
Hidden State Variability of Pretrained Language Models Can Guide Computation Reduction for Transfer Learning Shuo Xie Jiahao Qiu Ankita Pasad Li Du Qing Qu Hongyuan Mei 35 16 0 18 Oct 2022
The Lazy Neuron Phenomenon: On Emergence of Activation Sparsity in Transformers Zong-xiao Li Chong You Srinadh Bhojanapalli Daliang Li A. S. Rawat ... Kenneth Q Ye Felix Chern Felix X. Yu Ruiqi Guo Surinder Kumar MoE 27 87 0 12 Oct 2022
The Asymmetric Maximum Margin Bias of Quasi-Homogeneous Neural Networks D. Kunin Atsushi Yamamura Chao Ma Surya Ganguli 22 20 0 07 Oct 2022
Are All Losses Created Equal: A Neural Collapse Perspective Jinxin Zhou Chong You Xiao Li Kangning Liu Sheng Liu Qing Qu Zhihui Zhu 36 59 0 04 Oct 2022
FG-UAP: Feature-Gathering Universal Adversarial Perturbation Zhixing Ye Xinwen Cheng X. Huang AAML 74 10 0 27 Sep 2022
Neural Collapse with Normalized Features: A Geometric Analysis over the Riemannian Manifold Can Yaras Peng Wang Zhihui Zhu Laura Balzano Qing Qu 25 42 0 19 Sep 2022
Importance Tempering: Group Robustness for Overparameterized Models Yiping Lu Wenlong Ji Zachary Izzo Lexing Ying 45 7 0 19 Sep 2022
Linking Neural Collapse and L2 Normalization with Improved Out-of-Distribution Detection in Deep Neural Networks J. Haas William Yolland B. Rabus OODD 43 15 0 17 Sep 2022
Improving Self-Supervised Learning by Characterizing Idealized Representations Yann Dubois Tatsunori Hashimoto Stefano Ermon Percy Liang SSL 83 40 0 13 Sep 2022
Imbalance Trouble: Revisiting Neural-Collapse Geometry Christos Thrampoulidis Ganesh Ramachandra Kini V. Vakilian Tina Behnia 32 69 0 10 Aug 2022
Maximum Class Separation as Inductive Bias in One Matrix Tejaswi Kasarla Gertjan J. Burghouts Max van Spengler Elise van der Pol Rita Cucchiara Pascal Mettes 26 22 0 17 Jun 2022
Memorization-Dilation: Modeling Neural Collapse Under Label Noise Duc Anh Nguyen Ron Levie Julian Lienen Gitta Kutyniok Eyke Hüllermeier 16 1 0 11 Jun 2022
Neural Collapse: A Review on Modelling Principles and Generalization Vignesh Kothapalli 25 74 0 08 Jun 2022
On the Representation Collapse of Sparse Mixture of Experts Zewen Chi Li Dong Shaohan Huang Damai Dai Shuming Ma ... Payal Bajaj Xia Song Xian-Ling Mao Heyan Huang Furu Wei MoMe MoE 42 96 0 20 Apr 2022
Neural Collapse Inspired Attraction-Repulsion-Balanced Loss for Imbalanced Learning Liang Xie Yibo Yang Deng Cai Xiaofei He 21 44 0 19 Apr 2022
Inducing Neural Collapse in Imbalanced Learning: Do We Really Need a Learnable Classifier at the End of Deep Neural Network? Yibo Yang Shixiang Chen Hefei Ling Liangru Xie Zhouchen Lin Dacheng Tao 24 77 0 17 Mar 2022
On the Optimization Landscape of Neural Collapse under MSE Loss: Global Optimality with Unconstrained Features Jinxin Zhou Xiao Li Tian Ding Chong You Qing Qu Zhihui Zhu 30 99 0 02 Mar 2022
Robust Training under Label Noise by Over-parameterization Sheng Liu Zhihui Zhu Qing Qu Chong You NoLa OOD 27 106 0 28 Feb 2022
On the Implicit Bias Towards Minimal Depth of Deep Neural Networks Tomer Galanti Liane Galanti Ido Ben-Shaul 19 12 0 18 Feb 2022
Limitations of Neural Collapse for Understanding Generalization in Deep Learning Like Hui M. Belkin Preetum Nakkiran AI4CE 22 52 0 17 Feb 2022
Extended Unconstrained Features Model for Exploring Deep Neural Collapse Tom Tirer Joan Bruna AAML 16 95 0 16 Feb 2022
On Measuring Excess Capacity in Neural Networks Florian Graf Sebastian Zeng Bastian Alexander Rieck Marc Niethammer Roland Kwitt 19 10 0 16 Feb 2022
NeuroMixGDP: A Neural Collapse-Inspired Random Mixup for Private Data Release Donghao Li Yang Cao Yuan Yao 35 2 0 14 Feb 2022
Nearest Class-Center Simplification through Intermediate Layers Ido Ben-Shaul S. Dekel 42 26 0 21 Jan 2022
On the Role of Neural Collapse in Transfer Learning Tomer Galanti András Gyorgy Marcus Hutter SSL 21 87 0 30 Dec 2021
CoReS: Compatible Representations via Stationarity Niccoló Biondi F. Pernici Matteo Bruni A. Bimbo OOD 26 9 0 15 Nov 2021
An Unconstrained Layer-Peeled Perspective on Neural Collapse Wenlong Ji Yiping Lu Yiliang Zhang Zhun Deng Weijie J. Su 135 84 0 06 Oct 2021
Benign Overfitting in Multiclass Classification: All Roads Lead to Interpolation Ke Wang Vidya Muthukumar Christos Thrampoulidis 28 48 0 21 Jun 2021
Neural Collapse Under MSE Loss: Proximity to and Dynamics on the Central Path X. Y. Han Vardan Papyan D. Donoho AAML 33 141 0 03 Jun 2021
A Local Convergence Theory for Mildly Over-Parameterized Two-Layer Neural Network Mo Zhou Rong Ge Chi Jin 76 45 0 04 Feb 2021
Deep Networks and the Multiple Manifold Problem Sam Buchanan D. Gilboa John N. Wright 166 39 0 25 Aug 2020
From Symmetry to Geometry: Tractable Nonconvex Problems Yuqian Zhang Qing Qu John N. Wright 34 43 0 14 Jul 2020
Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks Lechao Xiao Yasaman Bahri Jascha Narain Sohl-Dickstein S. Schoenholz Jeffrey Pennington 244 349 0 14 Jun 2018
Global optimality conditions for deep neural networks Chulhee Yun S. Sra Ali Jadbabaie 128 117 0 08 Jul 2017