A Geometric Analysis of Neural Collapse with Unconstrained Features

6 May 2021

Qing Qu

Papers citing "A Geometric Analysis of Neural Collapse with Unconstrained Features"

50 / 148 papers shown

Title
Neural Multivariate Regression: Qualitative Insights from the Unconstrained Feature Model George Andriopoulos Soyuj Jung Basnet Juan Guevara Li Guo Keith Ross 30 0 0 14 May 2025
On the Geometry of Semantics in Next-token Prediction Yize Zhao Christos Thrampoulidis 23 0 0 13 May 2025
Enhancing Pre-Trained Model-Based Class-Incremental Learning through Neural Collapse Kun He Zijian Song Shuoxi Zhang J. Hopcroft CLL 56 0 0 25 Apr 2025
Learn by Reasoning: Analogical Weight Generation for Few-Shot Class-Incremental Learning Jizhou Han Chenhao Ding Yuhang He Songlin Dong Qiang Wang Xinyuan Gao Yihong Gong CLL 54 0 0 27 Mar 2025
An Overview of Low-Rank Structures in the Training and Adaptation of Large Models Laura Balzano Tianjiao Ding B. Haeffele Soo Min Kwon Qing Qu Peng Wang Zhilin Wang Can Yaras OffRL AI4CE 62 0 0 25 Mar 2025
Robust Weight Imprinting: Insights from Neural Collapse and Proxy-Based Aggregation Justus Westerhoff Golzar Atefi Mario Koddenbrock Alexei Figueroa Alexander Loser Erik Rodner Felix Alexader Gers OffRL 61 0 0 18 Mar 2025
A Good Start Matters: Enhancing Continual Learning with Data-Driven Weight Initialization Md Yousuf Harun Christopher Kanan AI4CE 50 0 0 09 Mar 2025
Position: Solve Layerwise Linear Models First to Understand Neural Dynamical Phenomena (Neural Collapse, Emergence, Lazy/Rich Regime, and Grokking) Yoonsoo Nam Seok Hyeong Lee Clementine Domine Yea Chan Park Charles London Wonyl Choi Niclas Goring Seungjai Lee AI4CE 38 0 0 28 Feb 2025
Implicit Geometry of Next-token Prediction: From Language Sparsity Patterns to Model Representations Yize Zhao Tina Behnia V. Vakilian Christos Thrampoulidis 55 8 0 20 Feb 2025
Learning with Noisy Labels: the Exploration of Error Bounds in Classification Haixia Liu Boxiao Li Can Yang Yang Wang 41 0 0 28 Jan 2025
Multi-layer Radial Basis Function Networks for Out-of-distribution Detection Amol Khanna Chenyi Ling Derek Everett Edward Raff Nathan Inkawhich OODD 33 0 0 05 Jan 2025
Understanding How Nonlinear Layers Create Linearly Separable Features for Low-Dimensional Data Alec S. Xu Can Yaras Peng Wang Q. Qu 30 0 0 04 Jan 2025
Neural Collapse Inspired Knowledge Distillation Shuoxi Zhang Zijian Song Kun He 69 1 0 16 Dec 2024
Dependence Induced Representations Xiangxiang Xu Lizhong Zheng 65 1 0 22 Nov 2024
Dual-Head Knowledge Distillation: Enhancing Logits Utilization with an Auxiliary Head Penghui Yang Chen-Chen Zong Sheng-Jun Huang Lei Feng Bo An 33 1 0 13 Nov 2024
Rethinking Decoders for Transformer-based Semantic Segmentation: A Compression Perspective Qishuai Wen Chun-Guang Li ViT 37 0 0 05 Nov 2024
Guiding Neural Collapse: Optimising Towards the Nearest Simplex Equiangular Tight Frame Evan Markou Thalaiyasingam Ajanthan Stephen Gould 31 0 0 02 Nov 2024
Weight decay induces low-rank attention layers Seijin Kobayashi Yassir Akram J. Oswald 39 6 0 31 Oct 2024
Understanding Representation of Deep Equilibrium Models from Neural Collapse Perspective Haixiang Sun Ye Shi 42 0 0 30 Oct 2024
The Persistence of Neural Collapse Despite Low-Rank Bias: An Analytic Perspective Through Unconstrained Features Connall Garrod Jonathan P. Keating 36 2 0 30 Oct 2024
Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization Noam Razin Sadhika Malladi Adithya Bhaskar Danqi Chen Sanjeev Arora Boris Hanin 91 16 0 11 Oct 2024
$$\ell_1$-norm rank-one symmetric matrix factorization has no spurious second-order stationary points$ $\ell_1$ -norm rank-one symmetric matrix factorization has no spurious second-order stationary points Jiewen Guan Anthony Man-Cho So 39 2 0 07 Oct 2024
Wide Neural Networks Trained with Weight Decay Provably Exhibit Neural Collapse Arthur Jacot Peter Súkeník Zihan Wang Marco Mondelli 31 1 0 07 Oct 2024
Control-oriented Clustering of Visual Latent Representation Han Qi Haocheng Yin Heng Yang SSL 50 2 0 07 Oct 2024
SGW-based Multi-Task Learning in Vision Tasks Ruiyuan Zhang Yuyao Chen Yuchi Huo Jiaxiang Liu Dianbing Xi Jie Liu Chao Wu 28 0 0 03 Oct 2024
The Prevalence of Neural Collapse in Neural Multivariate Regression George Andriopoulos Zixuan Dong Li Guo Zifan Zhao Keith Ross 49 3 0 06 Sep 2024
Beyond Unconstrained Features: Neural Collapse for Shallow Neural Networks with General Data Wanli Hong Shuyang Ling 42 3 0 03 Sep 2024
Geometric Analysis of Unconstrained Feature Models with $d=K$ Yi Shen Shao Gu 35 0 0 15 Jul 2024
Adaptive Parametric Activation Konstantinos Panagiotis Alexandridis Jiankang Deng Anh Nguyen Shan Luo 41 2 0 11 Jul 2024
LaCoOT: Layer Collapse through Optimal Transport Victor Quétu Nour Hezbri Enzo Tartaglione 34 0 0 13 Jun 2024
Towards Interpretable Deep Local Learning with Successive Gradient Reconciliation Yibo Yang Xiaojie Li Motasem Alfarra Hasan Hammoud Adel Bibi Philip H. S. Torr Guohao Li 37 2 0 07 Jun 2024
Federated Representation Learning in the Under-Parameterized Regime Renpu Liu Cong Shen Jing Yang 26 4 0 07 Jun 2024
Compressible Dynamics in Deep Overparameterized Low-Rank Learning & Adaptation Can Yaras Peng Wang Laura Balzano Qing Qu AI4CE 37 12 0 06 Jun 2024
Domain Game: Disentangle Anatomical Feature for Single Domain Generalized Segmentation Hao Chen Hongrun Zhang U. W. Chan Rui Yin Xiaofei Wang Chao Li OOD 42 1 0 04 Jun 2024
A Global Geometric Analysis of Maximal Coding Rate Reduction Peng Wang Huikang Liu Druv Pai Yaodong Yu Zhihui Zhu Q. Qu Yi Ma 34 6 0 04 Jun 2024
Federated Learning with Bilateral Curation for Partially Class-Disjoint Data Ziqing Fan Ruipeng Zhang Jiangchao Yao Bo Han Ya-Qin Zhang Yanfeng Wang FedML 37 12 0 29 May 2024
Bridging Mini-Batch and Asymptotic Analysis in Contrastive Learning: From InfoNCE to Kernel-Based Losses Panagiotis Koromilas Giorgos Bouritsas Theodoros Giannakopoulos M. Nicolaou Yannis Panagakis 40 0 0 28 May 2024
Linguistic Collapse: Neural Collapse in (Large) Language Models Robert Wu V. Papyan 48 12 0 28 May 2024
Neural Collapse versus Low-rank Bias: Is Deep Neural Collapse Really Optimal? Peter Súkeník Marco Mondelli Christoph H. Lampert AI4CE 34 7 0 23 May 2024
Improving Generalization of Deep Neural Networks by Optimum Shifting Yuyan Zhou Ye Li Lei Feng Sheng-Jun Huang OOD ODL 38 0 0 23 May 2024
Navigate Beyond Shortcuts: Debiased Learning through the Lens of Neural Collapse Yining Wang Junjie Sun Chenyue Wang Mi Zhang Min Yang 27 6 0 09 May 2024
Progressive Feedforward Collapse of ResNet Training Sicong Wang Kuo Gai Shihua Zhang AI4CE 33 4 0 02 May 2024
Unifying Low Dimensional Observations in Deep Learning Through the Deep Linear Unconstrained Feature Model Connall Garrod Jonathan P. Keating 41 8 0 09 Apr 2024
Can We Understand Plasticity Through Neural Collapse? Guglielmo Bonifazi Iason Chalas Gian Hess Jakub Lucki AI4CE 26 2 0 03 Apr 2024
Hyperspherical Classification with Dynamic Label-to-Prototype Assignment Mohammad Saeed Ebrahimi Saadabadi Ali Dabouei Sahar Rahimi Malakshan Nasser M. Nasrabadi 34 3 0 25 Mar 2024
Investigating the Benefits of Projection Head for Representation Learning Yihao Xue Eric Gan Jiayi Ni Siddharth Joshi Baharan Mirzasoleiman 47 10 0 18 Mar 2024
Supervised Contrastive Representation Learning: Landscape Analysis with Unconstrained Features Tina Behnia Christos Thrampoulidis SSL 36 0 0 29 Feb 2024
Deep Neural Network Models Trained With A Fixed Random Classifier Transfer Better Across Domains Hafiz Tiomoko Ali Umberto Michieli J. Moon Daehyun Kim Mete Ozay AAML 27 1 0 28 Feb 2024
Measuring and Controlling Instruction (In)Stability in Language Model Dialogs Kenneth Li Tianle Liu Naomi Bashkansky David Bau Fernanda Viégas Hanspeter Pfister Martin Wattenberg 18 6 0 13 Feb 2024
Low-Rank Learning by Design: the Role of Network Architecture and Activation Linearity in Gradient Rank Collapse Bradley T. Baker Ba Pearlmutter Robyn L. Miller Vince D. Calhoun Sergey Plis AI4CE 18 2 0 09 Feb 2024