Imbalance Trouble: Revisiting Neural-Collapse Geometry

10 August 2022

Christos Thrampoulidis

Ganesh Ramachandra Kini

V. Vakilian

Tina Behnia

ArXiv PDF HTML

Papers citing "Imbalance Trouble: Revisiting Neural-Collapse Geometry"

50 / 57 papers shown

Title
Neural Multivariate Regression: Qualitative Insights from the Unconstrained Feature Model George Andriopoulos Soyuj Jung Basnet Juan Guevara Li Guo Keith Ross 30 0 0 14 May 2025
On the Geometry of Semantics in Next-token Prediction Yize Zhao Christos Thrampoulidis 23 0 0 13 May 2025
Contextures: Representations from Contexts Runtian Zhai Kai Yang Che-Ping Tsai Burak Varici Zico Kolter Pradeep Ravikumar 143 0 0 02 May 2025
Enhancing Pre-Trained Model-Based Class-Incremental Learning through Neural Collapse Kun He Zijian Song Shuoxi Zhang J. Hopcroft CLL 56 0 0 25 Apr 2025
Balancing Two Classifiers via A Simplex ETF Structure for Model Calibration Jiani Ni He Zhao Jintong Gao Dandan Guo H. Zha 38 0 0 14 Apr 2025
Implicit Geometry of Next-token Prediction: From Language Sparsity Patterns to Model Representations Yize Zhao Tina Behnia V. Vakilian Christos Thrampoulidis 60 8 0 20 Feb 2025
Learning with Noisy Labels: the Exploration of Error Bounds in Classification Haixia Liu Boxiao Li Can Yang Yang Wang 41 0 0 28 Jan 2025
Guiding Neural Collapse: Optimising Towards the Nearest Simplex Equiangular Tight Frame Evan Markou Thalaiyasingam Ajanthan Stephen Gould 31 0 0 02 Nov 2024
Understanding Representation of Deep Equilibrium Models from Neural Collapse Perspective Haixiang Sun Ye Shi 42 0 0 30 Oct 2024
The Persistence of Neural Collapse Despite Low-Rank Bias: An Analytic Perspective Through Unconstrained Features Connall Garrod Jonathan P. Keating 36 2 0 30 Oct 2024
The Fair Language Model Paradox Andrea Pinto Tomer Galanti Randall Balestriero 25 0 0 15 Oct 2024
Wide Neural Networks Trained with Weight Decay Provably Exhibit Neural Collapse Arthur Jacot Peter Súkeník Zihan Wang Marco Mondelli 31 1 0 07 Oct 2024
Multiplicative Logit Adjustment Approximates Neural-Collapse-Aware Decision Boundary Adjustment Naoya Hasegawa Issei Sato 41 0 0 26 Sep 2024
The Prevalence of Neural Collapse in Neural Multivariate Regression George Andriopoulos Zixuan Dong Li Guo Zifan Zhao Keith Ross 52 3 0 06 Sep 2024
Beyond Unconstrained Features: Neural Collapse for Shallow Neural Networks with General Data Wanli Hong Shuyang Ling 45 3 0 03 Sep 2024
All-around Neural Collapse for Imbalanced Classification Enhao Zhang Chaohua Li Chuanxing Geng Songcan Chen AI4CE 36 1 0 14 Aug 2024
Bridging Mini-Batch and Asymptotic Analysis in Contrastive Learning: From InfoNCE to Kernel-Based Losses Panagiotis Koromilas Giorgos Bouritsas Theodoros Giannakopoulos M. Nicolaou Yannis Panagakis 40 0 0 28 May 2024
Linguistic Collapse: Neural Collapse in (Large) Language Models Robert Wu Vardan Papyan 48 12 0 28 May 2024
Neural Collapse versus Low-rank Bias: Is Deep Neural Collapse Really Optimal? Peter Súkeník Marco Mondelli Christoph H. Lampert AI4CE 34 7 0 23 May 2024
Navigate Beyond Shortcuts: Debiased Learning through the Lens of Neural Collapse Yining Wang Junjie Sun Chenyue Wang Mi Zhang Min Yang 27 6 0 09 May 2024
Unifying Low Dimensional Observations in Deep Learning Through the Deep Linear Unconstrained Feature Model Connall Garrod Jonathan P. Keating 41 8 0 09 Apr 2024
Accurately Predicting Probabilities of Safety-Critical Rare Events for Intelligent Systems Ruoxuan Bai Jingxuan Yang Weiduo Gong Yi Zhang Qiujing Lu Shuo Feng 55 1 0 20 Mar 2024
Supervised Contrastive Representation Learning: Landscape Analysis with Unconstrained Features Tina Behnia Christos Thrampoulidis SSL 39 0 0 29 Feb 2024
Average gradient outer product as a mechanism for deep neural collapse Daniel Beaglehole Peter Súkeník Marco Mondelli Misha Belkin AI4CE 40 12 0 21 Feb 2024
Low-Rank Learning by Design: the Role of Network Architecture and Activation Linearity in Gradient Rank Collapse Bradley T. Baker Ba Pearlmutter Robyn L. Miller Vince D. Calhoun Sergey Plis AI4CE 23 2 0 09 Feb 2024
Cross Entropy versus Label Smoothing: A Neural Collapse Perspective Li Guo Keith Ross Zifan Zhao George Andriopoulos Shuyang Ling Yufeng Xu Zixuan Dong UQCV NoLa 30 9 0 06 Feb 2024
Enhancing Compositional Generalization via Compositional Feature Alignment Haoxiang Wang Haozhe Si Huajie Shao Han Zhao 40 1 0 05 Feb 2024
FedLoGe: Joint Local and Generic Federated Learning under Long-tailed Data Zikai Xiao Zihan Chen Liyinglan Liu Yang Feng Jian Wu Wanlu Liu Qiufeng Wang Howard H. Yang Zuo-Qiang Liu FedML 39 6 0 17 Jan 2024
Neural Collapse for Cross-entropy Class-Imbalanced Learning with Unconstrained ReLU Feature Model Hien Dang Tho Tran Tan Minh Nguyen Nhat Ho 22 11 0 04 Jan 2024
Neural Collapse in Multi-label Learning with Pick-all-label Loss Pengyu Li Xiao Li Yutong Wang Qing Qu 30 8 0 24 Oct 2023
It's an Alignment, Not a Trade-off: Revisiting Bias and Variance in Deep Models Lin Chen Michal Lukasik Wittawat Jitkrittum Chong You Sanjiv Kumar 40 2 0 13 Oct 2023
Generalized Neural Collapse for a Large Number of Classes Jiachen Jiang Jinxin Zhou Peng Wang Qing Qu Dustin Mixon Chong You Zhihui Zhu AI4CE 34 24 0 09 Oct 2023
Engineering the Neural Collapse Geometry of Supervised-Contrastive Loss Jaidev Gill V. Vakilian Christos Thrampoulidis 23 1 0 02 Oct 2023
Investigating the Catastrophic Forgetting in Multimodal Large Language Models Yuexiang Zhai Shengbang Tong Xiao Li Mu Cai Qing Qu Yong Jae Lee Y. Ma VLM MLLM CLL 77 77 0 19 Sep 2023
Neural Collapse for Unconstrained Feature Model under Cross-entropy Loss with Imbalanced Data Wanli Hong Shuyang Ling 28 14 0 18 Sep 2023
Transformers as Support Vector Machines Davoud Ataee Tarzanagh Yingcong Li Christos Thrampoulidis Samet Oymak 48 43 0 31 Aug 2023
Neural Collapse Terminus: A Unified Solution for Class Incremental Learning and Its Variants Yibo Yang Haobo Yuan Hefei Ling Jianlong Wu Lefei Zhang Zhouchen Lin Philip Torr Dacheng Tao Guohao Li CLL 31 8 0 03 Aug 2023
$Few-shot $\mathbf{1/a}$ Anomalies Feedback : Damage Vision Mining Opportunity and Embedding Feature Imbalance$ Few-shot $\mathbf{1/a}$ Anomalies Feedback : Damage Vision Mining Opportunity and Embedding Feature Imbalance Takato Yasuno AI4TS 23 0 0 24 Jul 2023
A Neural Collapse Perspective on Feature Evolution in Graph Neural Networks Vignesh Kothapalli Tom Tirer Joan Bruna 38 11 0 04 Jul 2023
Understanding Prompt Tuning for V-L Models Through the Lens of Neural Collapse Didi Zhu Zexi Li Min Zhang Junkun Yuan Yunfeng Shao Jiashuo Liu Kun Kuang Yinchuan Li Chao Wu VLM 26 1 0 28 Jun 2023
Symmetric Neural-Collapse Representations with Supervised Contrastive Loss: The Impact of ReLU and Batching Ganesh Ramachandra Kini V. Vakilian Tina Behnia Jaidev Gill Christos Thrampoulidis 17 1 0 13 Jun 2023
Quantifying the Variability Collapse of Neural Networks Jing-Xue Xu Haoxiong Liu 36 5 0 06 Jun 2023
Exploring Weight Balancing on Long-Tailed Recognition Problem Naoya Hasegawa Issei Sato 27 6 0 26 May 2023
Deep Neural Collapse Is Provably Optimal for the Deep Unconstrained Features Model Peter Súkeník Marco Mondelli Christoph H. Lampert 26 21 0 22 May 2023
No Fear of Classifier Biases: Neural Collapse Inspired Federated Learning with Synthetic and Fixed Classifier Zexi Li Xinyi Shang Rui He Tao R. Lin Chao Wu FedML 47 54 0 17 Mar 2023
On the Implicit Geometry of Cross-Entropy Parameterizations for Label-Imbalanced Data Tina Behnia Ganesh Ramachandra Kini V. Vakilian Christos Thrampoulidis 44 17 0 14 Mar 2023
Generalizing and Decoupling Neural Collapse via Hyperspherical Uniformity Gap Weiyang Liu L. Yu Adrian Weller Bernhard Schölkopf 37 18 0 11 Mar 2023
Inducing Neural Collapse in Deep Long-tailed Learning Xuantong Liu Jianfeng Zhang Tianyang Hu He Cao Lujia Pan Yuan Yao 27 27 0 24 Feb 2023
Learning Prototype Classifiers for Long-Tailed Recognition Saurabh Sharma Yongqin Xian Ning Yu Ambuj K. Singh 37 13 0 01 Feb 2023
Understanding Imbalanced Semantic Segmentation Through Neural Collapse Zhisheng Zhong Jiequan Cui Yibo Yang Xiaoyang Wu Xiaojuan Qi Xinming Zhang Jiaya Jia 138 45 0 03 Jan 2023