A Geometric Analysis of Neural Collapse with Unconstrained Features

6 May 2021

Qing Qu

Papers citing "A Geometric Analysis of Neural Collapse with Unconstrained Features"

50 / 149 papers shown

Title
Sharper Convergence Rates for Nonconvex Optimisation via Reduction Mappings Evan Markou Thalaiyasingam Ajanthan Stephen Gould 35 0 0 10 Jun 2025
Neural Collapse in Cumulative Link Models for Ordinal Regression: An Analysis with Unconstrained Feature Model Chuang Ma Tomoyuki Obuchi Toshiyuki Tanaka 54 0 0 06 Jun 2025
BioCLIP 2: Emergent Properties from Scaling Hierarchical Contrastive Learning Zanming Huang Samuel Stevens Elizabeth G. Campolongo Matthew J. Thompson Net Zhang ... Daniel Rubenstein Hilmar Lapp T. Berger-Wolf Wei-Lun Chao Yu-Chuan Su VLM 61 2 0 29 May 2025
From Compression to Expansion: A Layerwise Analysis of In-Context Learning Jiachen Jiang Yuxin Dong Jinxin Zhou Zhihui Zhu 65 0 0 22 May 2025
Neural Collapse is Globally Optimal in Deep Regularized ResNets and Transformers Peter Súkeník Christoph H. Lampert Marco Mondelli 79 0 0 21 May 2025
Neural Multivariate Regression: Qualitative Insights from the Unconstrained Feature Model Shuyang Ling Soyuj Jung Basnet Juan Guevara Li Guo George Andriopoulos 81 0 0 14 May 2025
On the Geometry of Semantics in Next-token Prediction Yize Zhao Christos Thrampoulidis 57 0 0 13 May 2025
Enhancing Pre-Trained Model-Based Class-Incremental Learning through Neural Collapse Kun He Zijian Song Shuoxi Zhang John E. Hopcroft CLL 145 0 0 25 Apr 2025
Learn by Reasoning: Analogical Weight Generation for Few-Shot Class-Incremental Learning Jizhou Han Chenhao Ding Yuhang He Songlin Dong Qiang Wang Xinyuan Gao Yihong Gong CLL 105 0 0 27 Mar 2025
An Overview of Low-Rank Structures in the Training and Adaptation of Large Models Laura Balzano Tianjiao Ding B. Haeffele Soo Min Kwon Qing Qu Peng Wang Ziyi Wang Can Yaras OffRL AI4CE 88 2 0 25 Mar 2025
Robust Weight Imprinting: Insights from Neural Collapse and Proxy-Based Aggregation Justus Westerhoff Golzar Atefi Mario Koddenbrock Alexei Figueroa Alexander Loser Erik Rodner Felix Alexader Gers OffRL 112 0 0 18 Mar 2025
A Good Start Matters: Enhancing Continual Learning with Data-Driven Weight Initialization Md Yousuf Harun Christopher Kanan AI4CE 93 0 0 09 Mar 2025
Position: Solve Layerwise Linear Models First to Understand Neural Dynamical Phenomena (Neural Collapse, Emergence, Lazy/Rich Regime, and Grokking) Yoonsoo Nam Seok Hyeong Lee Clementine Domine Yea Chan Park Charles London Wonyl Choi Niclas Goring Seungjai Lee AI4CE 221 1 0 28 Feb 2025
Implicit Geometry of Next-token Prediction: From Language Sparsity Patterns to Model Representations Yize Zhao Tina Behnia V. Vakilian Christos Thrampoulidis 195 10 0 20 Feb 2025
The Exploration of Error Bounds in Classification with Noisy Labels Haixia Liu Boxiao Li Can Yang Yang Wang NoLa 178 0 0 25 Jan 2025
Multi-layer Radial Basis Function Networks for Out-of-distribution Detection Amol Khanna Chenyi Ling Derek Everett Edward Raff Nathan Inkawhich OODD 127 0 0 05 Jan 2025
Understanding How Nonlinear Layers Create Linearly Separable Features for Low-Dimensional Data Alec S. Xu Can Yaras Peng Wang Q. Qu 97 1 0 04 Jan 2025
Neural Collapse Inspired Knowledge Distillation Shuoxi Zhang Zijian Song Kun He 181 1 0 16 Dec 2024
Explaining and Mitigating the Modality Gap in Contrastive Multimodal Learning Can Yaras Siyi Chen Peng Wang Q. Qu VLM 96 3 0 10 Dec 2024
Dependence Induced Representations Xiangxiang Xu Lizhong Zheng 111 1 0 22 Nov 2024
Dual-Head Knowledge Distillation: Enhancing Logits Utilization with an Auxiliary Head Penghui Yang Chen-Chen Zong Sheng-Jun Huang Lei Feng Bo An 145 1 0 13 Nov 2024
Rethinking Decoders for Transformer-based Semantic Segmentation: A Compression Perspective Qishuai Wen Chun-Guang Li ViT 70 0 0 05 Nov 2024
Guiding Neural Collapse: Optimising Towards the Nearest Simplex Equiangular Tight Frame Evan Markou Thalaiyasingam Ajanthan Stephen Gould 97 3 0 02 Nov 2024
Weight decay induces low-rank attention layers Seijin Kobayashi Yassir Akram J. Oswald 122 12 0 31 Oct 2024
Understanding Representation of Deep Equilibrium Models from Neural Collapse Perspective Haixiang Sun Ye Shi 96 0 0 30 Oct 2024
The Persistence of Neural Collapse Despite Low-Rank Bias: An Analytic Perspective Through Unconstrained Features Connall Garrod Jonathan P. Keating 67 4 0 30 Oct 2024
Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization Noam Razin Sadhika Malladi Adithya Bhaskar Danqi Chen Sanjeev Arora Boris Hanin 223 35 0 11 Oct 2024
$$\ell_1$-norm rank-one symmetric matrix factorization has no spurious second-order stationary points$ $\ell_1$ -norm rank-one symmetric matrix factorization has no spurious second-order stationary points Jiewen Guan Anthony Man-Cho So 111 2 0 07 Oct 2024
Wide Neural Networks Trained with Weight Decay Provably Exhibit Neural Collapse Arthur Jacot Peter Súkeník Zihan Wang Marco Mondelli 74 2 0 07 Oct 2024
Control-oriented Clustering of Visual Latent Representation Han Qi Haocheng Yin Heng Yang SSL 145 2 0 07 Oct 2024
SGW-based Multi-Task Learning in Vision Tasks Ruiyuan Zhang Yuyao Chen Yuchi Huo Jiaxiang Liu Dianbing Xi Jie Liu Chao Wu 83 1 0 03 Oct 2024
The Prevalence of Neural Collapse in Neural Multivariate Regression Shuyang Ling Zixuan Dong Li Guo Zifan Zhao George Andriopoulos 82 7 0 06 Sep 2024
Beyond Unconstrained Features: Neural Collapse for Shallow Neural Networks with General Data Wanli Hong Shuyang Ling 71 4 0 03 Sep 2024
Geometric Analysis of Unconstrained Feature Models with $d=K$ Yi Shen Shao Gu 59 0 0 15 Jul 2024
Adaptive Parametric Activation Konstantinos Panagiotis Alexandridis Jiankang Deng Anh Nguyen Shan Luo 86 5 0 11 Jul 2024
LaCoOT: Layer Collapse through Optimal Transport Victor Quétu Nour Hezbri Enzo Tartaglione 66 0 0 13 Jun 2024
Towards Interpretable Deep Local Learning with Successive Gradient Reconciliation Yibo Yang Xiaojie Li Motasem Alfarra Hasan Hammoud Adel Bibi Philip Torr Guohao Li 84 3 0 07 Jun 2024
Federated Representation Learning in the Under-Parameterized Regime Renpu Liu Cong Shen Jing Yang 90 5 0 07 Jun 2024
Compressible Dynamics in Deep Overparameterized Low-Rank Learning & Adaptation Can Yaras Peng Wang Laura Balzano Qing Qu AI4CE 73 15 0 06 Jun 2024
Domain Game: Disentangle Anatomical Feature for Single Domain Generalized Segmentation Hao Chen Hongrun Zhang U. W. Chan Rui Yin Xiaofei Wang Chao Li OOD 84 1 0 04 Jun 2024
A Global Geometric Analysis of Maximal Coding Rate Reduction Peng Wang Huikang Liu Druv Pai Yaodong Yu Zhihui Zhu Q. Qu Yi-An Ma 68 7 0 04 Jun 2024
Federated Learning with Bilateral Curation for Partially Class-Disjoint Data Ziqing Fan Ruipeng Zhang Jiangchao Yao Bo Han Ya Zhang Yanfeng Wang FedML 73 13 0 29 May 2024
Bridging Mini-Batch and Asymptotic Analysis in Contrastive Learning: From InfoNCE to Kernel-Based Losses Panagiotis Koromilas Giorgos Bouritsas Theodoros Giannakopoulos M. Nicolaou Yannis Panagakis 77 2 0 28 May 2024
Linguistic Collapse: Neural Collapse in (Large) Language Models Robert Wu Vardan Papyan 97 16 0 28 May 2024
Neural Collapse versus Low-rank Bias: Is Deep Neural Collapse Really Optimal? Peter Súkeník Marco Mondelli Christoph H. Lampert AI4CE 74 11 0 23 May 2024
Improving Generalization of Deep Neural Networks by Optimum Shifting Yuyan Zhou Ye Li Lei Feng Sheng-Jun Huang OOD ODL 55 0 0 23 May 2024
Navigate Beyond Shortcuts: Debiased Learning through the Lens of Neural Collapse Yining Wang Junjie Sun Chenyue Wang Mi Zhang Min Yang 87 7 0 09 May 2024
Progressive Feedforward Collapse of ResNet Training Sicong Wang Kuo Gai Shihua Zhang AI4CE 78 7 0 02 May 2024
Unifying Low Dimensional Observations in Deep Learning Through the Deep Linear Unconstrained Feature Model Connall Garrod Jonathan P. Keating 117 9 0 09 Apr 2024
Can We Understand Plasticity Through Neural Collapse? Guglielmo Bonifazi Iason Chalas Gian Hess Jakub Lucki AI4CE 105 2 0 03 Apr 2024