A Geometric Analysis of Neural Collapse with Unconstrained Features

6 May 2021

Qing Qu

Papers citing "A Geometric Analysis of Neural Collapse with Unconstrained Features"

49 / 149 papers shown

Title
Feature Collapse T. Laurent J. V. Brecht Xavier Bresson 75 3 0 25 May 2023
Deep Neural Collapse Is Provably Optimal for the Deep Unconstrained Features Model Peter Súkeník Marco Mondelli Christoph H. Lampert 63 27 0 22 May 2023
Towards understanding neural collapse in supervised contrastive learning with the information bottleneck method Siwei Wang S. Palmer 104 2 0 19 May 2023
A Study of Neural Collapse Phenomenon: Grassmannian Frame, Symmetry and Generalization Peifeng Gao Qianqian Xu Peisong Wen Huiyang Shao Zhiyong Yang Qingming Huang 71 6 0 18 Apr 2023
Neural Collapse Inspired Federated Learning with Non-iid Data Chenxi Huang Liang Xie Yibo Yang Wenxiao Wang Binbin Lin D. Cai FedML 127 3 0 27 Mar 2023
Factorizers for Distributed Sparse Block Codes Michael Hersche Aleksandar Terzić G. Karunaratne Jovin Langenegger Angeline Pouget G. Cherubini Luca Benini Abu Sebastian Abbas Rahimi 110 4 0 24 Mar 2023
No Fear of Classifier Biases: Neural Collapse Inspired Federated Learning with Synthetic and Fixed Classifier Zexi Li Xinyi Shang Rui He Tao R. Lin Chao Wu FedML 123 55 0 17 Mar 2023
On the Implicit Geometry of Cross-Entropy Parameterizations for Label-Imbalanced Data Tina Behnia Ganesh Ramachandra Kini V. Vakilian Christos Thrampoulidis 106 17 0 14 Mar 2023
Generalizing and Decoupling Neural Collapse via Hyperspherical Uniformity Gap Weiyang Liu L. Yu Adrian Weller Bernhard Schölkopf 102 18 0 11 Mar 2023
A Message Passing Perspective on Learning Dynamics of Contrastive Learning Yifei Wang Qi Zhang Tianqi Du Jiansheng Yang Zhouchen Lin Yisen Wang SSL 67 20 0 08 Mar 2023
Inducing Neural Collapse in Deep Long-tailed Learning Xuantong Liu Jianfeng Zhang Tianyang Hu He Cao Lujia Pan Yuan Yao 73 31 0 24 Feb 2023
Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class Incremental Learning Yibo Yang Haobo Yuan Hefei Ling Zhouchen Lin Philip Torr Dacheng Tao CLL 111 108 0 06 Feb 2023
Unsupervised Manifold Linearizing and Clustering Tianjiao Ding Shengbang Tong Kwan Ho Ryan Chan Xili Dai Yi Ma B. Haeffele 96 12 0 04 Jan 2023
Understanding Imbalanced Semantic Segmentation Through Neural Collapse Zhisheng Zhong Jiequan Cui Yibo Yang Xiaoyang Wu Xiaojuan Qi Xinming Zhang Jiaya Jia 221 52 0 03 Jan 2023
Pruning Before Training May Improve Generalization, Provably Hongru Yang Yingbin Liang Xiaojie Guo Lingfei Wu Zhangyang Wang MLT 64 2 0 01 Jan 2023
Generalization Bounds for Few-Shot Transfer Learning with Pretrained Classifiers Tomer Galanti András Gyorgy Marcus Hutter VLM SSL 67 4 0 23 Dec 2022
Principled and Efficient Transfer Learning of Deep Models via Neural Collapse Xiao Li Sheng Liu Jin-li Zhou Xin Lu C. Fernandez‐Granda Zhihui Zhu Q. Qu AAML 97 20 0 23 Dec 2022
Perturbation Analysis of Neural Collapse Tom Tirer Haoxiang Huang Jonathan Niles-Weed AAML 82 26 0 29 Oct 2022
Grokking phase transitions in learning local rules with gradient descent Bojan Žunkovič E. Ilievski 105 17 0 26 Oct 2022
Hidden State Variability of Pretrained Language Models Can Guide Computation Reduction for Transfer Learning Shuo Xie Jiahao Qiu Ankita Pasad Li Du Qing Qu Hongyuan Mei 87 16 0 18 Oct 2022
The Lazy Neuron Phenomenon: On Emergence of Activation Sparsity in Transformers Zong-xiao Li Chong You Srinadh Bhojanapalli Daliang Li A. S. Rawat ... Kenneth Q Ye Felix Chern Felix X. Yu Ruiqi Guo Surinder Kumar MoE 106 97 0 12 Oct 2022
The Asymmetric Maximum Margin Bias of Quasi-Homogeneous Neural Networks D. Kunin Atsushi Yamamura Chao Ma Surya Ganguli 79 21 0 07 Oct 2022
Are All Losses Created Equal: A Neural Collapse Perspective Jinxin Zhou Chong You Xiao Li Kangning Liu Sheng Liu Qing Qu Zhihui Zhu 78 65 0 04 Oct 2022
FG-UAP: Feature-Gathering Universal Adversarial Perturbation Zhixing Ye Xinwen Cheng Xiaolin Huang AAML 110 11 0 27 Sep 2022
Neural Collapse with Normalized Features: A Geometric Analysis over the Riemannian Manifold Can Yaras Peng Wang Zhihui Zhu Laura Balzano Qing Qu 66 44 0 19 Sep 2022
Importance Tempering: Group Robustness for Overparameterized Models Yiping Lu Wenlong Ji Zachary Izzo Lexing Ying 87 7 0 19 Sep 2022
Linking Neural Collapse and L2 Normalization with Improved Out-of-Distribution Detection in Deep Neural Networks J. Haas William Yolland B. Rabus OODD 103 20 0 17 Sep 2022
Improving Self-Supervised Learning by Characterizing Idealized Representations Yann Dubois Tatsunori Hashimoto Stefano Ermon Percy Liang SSL 167 43 0 13 Sep 2022
Imbalance Trouble: Revisiting Neural-Collapse Geometry Christos Thrampoulidis Ganesh Ramachandra Kini V. Vakilian Tina Behnia 69 72 0 10 Aug 2022
Maximum Class Separation as Inductive Bias in One Matrix Tejaswi Kasarla Gertjan J. Burghouts Max van Spengler Elise van der Pol Rita Cucchiara Pascal Mettes 87 22 0 17 Jun 2022
Memorization-Dilation: Modeling Neural Collapse Under Label Noise Duc Anh Nguyen Ron Levie Julian Lienen Gitta Kutyniok Eyke Hüllermeier 67 2 0 11 Jun 2022
Neural Collapse: A Review on Modelling Principles and Generalization Vignesh Kothapalli 158 82 0 08 Jun 2022
On the Representation Collapse of Sparse Mixture of Experts Zewen Chi Li Dong Shaohan Huang Damai Dai Shuming Ma ... Payal Bajaj Xia Song Xian-Ling Mao Heyan Huang Furu Wei MoMe MoE 117 106 0 20 Apr 2022
Neural Collapse Inspired Attraction-Repulsion-Balanced Loss for Imbalanced Learning Liang Xie Yibo Yang Deng Cai Xiaofei He 65 47 0 19 Apr 2022
Inducing Neural Collapse in Imbalanced Learning: Do We Really Need a Learnable Classifier at the End of Deep Neural Network? Yibo Yang Shixiang Chen Hefei Ling Liangru Xie Zhouchen Lin Dacheng Tao 107 83 0 17 Mar 2022
On the Optimization Landscape of Neural Collapse under MSE Loss: Global Optimality with Unconstrained Features Jinxin Zhou Xiao Li Tian Ding Chong You Qing Qu Zhihui Zhu 78 102 0 02 Mar 2022
Robust Training under Label Noise by Over-parameterization Sheng Liu Zhihui Zhu Qing Qu Chong You NoLa OOD 77 109 0 28 Feb 2022
On the Implicit Bias Towards Minimal Depth of Deep Neural Networks Tomer Galanti Liane Galanti Ido Ben-Shaul 52 14 0 18 Feb 2022
Limitations of Neural Collapse for Understanding Generalization in Deep Learning Like Hui M. Belkin Preetum Nakkiran AI4CE 65 56 0 17 Feb 2022
Extended Unconstrained Features Model for Exploring Deep Neural Collapse Tom Tirer Joan Bruna AAML 63 101 0 16 Feb 2022
On Measuring Excess Capacity in Neural Networks Florian Graf Sebastian Zeng Bastian Rieck Marc Niethammer Roland Kwitt 105 10 0 16 Feb 2022
NeuroMixGDP: A Neural Collapse-Inspired Random Mixup for Private Data Release Donghao Li Yang Cao Yuan Yao 85 2 0 14 Feb 2022
Nearest Class-Center Simplification through Intermediate Layers Ido Ben-Shaul S. Dekel 95 27 0 21 Jan 2022
On the Role of Neural Collapse in Transfer Learning Tomer Galanti András Gyorgy Marcus Hutter SSL 90 93 0 30 Dec 2021
CoReS: Compatible Representations via Stationarity Niccoló Biondi F. Pernici Matteo Bruni A. Bimbo OOD 67 9 0 15 Nov 2021
An Unconstrained Layer-Peeled Perspective on Neural Collapse Wenlong Ji Yiping Lu Yiliang Zhang Zhun Deng Weijie J. Su 212 87 0 06 Oct 2021
Benign Overfitting in Multiclass Classification: All Roads Lead to Interpolation Ke Wang Vidya Muthukumar Christos Thrampoulidis 115 49 0 21 Jun 2021
Neural Collapse Under MSE Loss: Proximity to and Dynamics on the Central Path X. Y. Han Vardan Papyan D. Donoho AAML 139 144 0 03 Jun 2021
From Symmetry to Geometry: Tractable Nonconvex Problems Yuqian Zhang Qing Qu John N. Wright 89 45 0 14 Jul 2020