Prevalence of Neural Collapse during the terminal phase of deep learning training

18 August 2020

Papers citing "Prevalence of Neural Collapse during the terminal phase of deep learning training"

50 / 127 papers shown

Title
Neural Multivariate Regression: Qualitative Insights from the Unconstrained Feature Model George Andriopoulos Soyuj Jung Basnet Juan Guevara Li Guo Keith Ross 30 0 0 14 May 2025
$Simple Semi-supervised Knowledge Distillation from Vision-Language Models via $\mathbf{\texttt{D}}$ual-$\mathbf{\texttt{H}}$ead $\mathbf{\texttt{O}}$ptimization$ Simple Semi-supervised Knowledge Distillation from Vision-Language Models via $\mathbf{\texttt{D}}$ ual- $\mathbf{\texttt{H}}$ ead $\mathbf{\texttt{O}}$ ptimization Seongjae Kang Dong Bok Lee Hyungjoon Jang Sung Ju Hwang VLM 57 0 0 12 May 2025
Crowding Out The Noise: Algorithmic Collective Action Under Differential Privacy Rushabh Solanki Meghana Bhange Ulrich Aïvodji Elliot Creager 29 0 0 09 May 2025
Register and CLS tokens yield a decoupling of local and global features in large ViTs Alexander Lappe M. Giese 24 0 0 09 May 2025
The Evolution of Embedding Table Optimization and Multi-Epoch Training in Pinterest Ads Conversion Andrew Qiu Shubham Barhate Hin Wai Lui Runze Su Rafael Rios Müller Kungang Li Ling Leng Han Sun Shayan Ehsani Zhifang Liu 31 0 0 08 May 2025
Contextures: Representations from Contexts Runtian Zhai Kai Yang Che-Ping Tsai Burak Varici Zico Kolter Pradeep Ravikumar 137 0 0 02 May 2025
Enhancing Pre-Trained Model-Based Class-Incremental Learning through Neural Collapse Kun He Zijian Song Shuoxi Zhang J. Hopcroft CLL 56 0 0 25 Apr 2025
You Are Your Own Best Teacher: Achieving Centralized-level Performance in Federated Learning under Heterogeneous and Long-tailed Data Shanshan Yan Zexi Li Chao-Xiang Wu Meng Pang Yang Lu Yan Yan Hanzi Wang FedML 61 0 0 10 Mar 2025
Learning from End User Data with Shuffled Differential Privacy over Kernel Densities Tal Wagner FedML 53 0 0 21 Feb 2025
Implicit Geometry of Next-token Prediction: From Language Sparsity Patterns to Model Representations Yize Zhao Tina Behnia V. Vakilian Christos Thrampoulidis 55 8 0 20 Feb 2025
A Relative Homology Theory of Representation in Neural Networks Kosio Beshkov 99 0 0 17 Feb 2025
Position: Curvature Matrices Should Be Democratized via Linear Operators Felix Dangel Runa Eschenhagen Weronika Ormaniec Andres Fernandez Lukas Tatzel Agustinus Kristiadi 58 3 0 31 Jan 2025
Technical report on label-informed logit redistribution for better domain generalization in low-shot classification with foundation models Behraj Khan T. Syed 151 1 0 29 Jan 2025
Fresh-CL: Feature Realignment through Experts on Hypersphere in Continual Learning Zhongyi Zhou Yaxin Peng Pin Yi Minjie Zhu Chaomin Shen 151 0 0 04 Jan 2025
Measuring Error Alignment for Decision-Making Systems Binxia Xu Antonis Bikakis Daniel Onah A. Vlachidis Luke Dickens 41 0 0 03 Jan 2025
CLOSER: Towards Better Representation Learning for Few-Shot Class-Incremental Learning Junghun Oh Sungyong Baik Kyoung Mu Lee CLL 34 3 0 08 Oct 2024
Control-oriented Clustering of Visual Latent Representation Han Qi Haocheng Yin Heng Yang SSL 55 2 0 07 Oct 2024
Collapsed Language Models Promote Fairness Jingxuan Xu Wuyang Chen Linyi Li Yao Zhao Yunchao Wei 46 0 0 06 Oct 2024
SGW-based Multi-Task Learning in Vision Tasks Ruiyuan Zhang Yuyao Chen Yuchi Huo Jiaxiang Liu Dianbing Xi Jie Liu Chao Wu 28 0 0 03 Oct 2024
Formation of Representations in Neural Networks Liu Ziyin Isaac Chuang Tomer Galanti T. Poggio 39 4 0 03 Oct 2024
Linear Projections of Teacher Embeddings for Few-Class Distillation Noel Loo Fotis Iliopoulos Wei Hu Erik Vee 30 0 0 30 Sep 2024
Multiplicative Logit Adjustment Approximates Neural-Collapse-Aware Decision Boundary Adjustment Naoya Hasegawa Issei Sato 41 0 0 26 Sep 2024
Exploring Information-Theoretic Metrics Associated with Neural Collapse in Supervised Training Kun Song Zhiquan Tan Bochao Zou Jiansheng Chen Huimin Ma Weiran Huang 42 0 0 25 Sep 2024
When resampling/reweighting improves feature learning in imbalanced classification?: A toy-model study Tomoyuki Obuchi Toshiyuki Tanaka 46 0 0 09 Sep 2024
Prototypical Partial Optimal Transport for Universal Domain Adaptation Yucheng Yang Xiang Gu Jian Sun OT 49 11 0 02 Aug 2024
Novel Deep Neural Network Classifier Characterization Metrics with Applications to Dataless Evaluation Nathaniel R. Dean Dilip Sarkar 35 0 0 17 Jul 2024
ProSub: Probabilistic Open-Set Semi-Supervised Learning with Subspace-Based Out-of-Distribution Detection Erik Wallin Lennart Svensson Fredrik Kahl Lars Hammarstrand OODD 42 1 0 16 Jul 2024
Geometric Analysis of Unconstrained Feature Models with $d=K$ Yi Shen Shao Gu 35 0 0 15 Jul 2024
Normalization and effective learning rates in reinforcement learning Clare Lyle Zeyu Zheng Khimya Khetarpal James Martens H. V. Hasselt Razvan Pascanu Will Dabney 19 7 0 01 Jul 2024
Cross-Architecture Auxiliary Feature Space Translation for Efficient Few-Shot Personalized Object Detection F. Barbato Umberto Michieli J. Moon Pietro Zanuttigh Mete Ozay 42 2 0 01 Jul 2024
A deep neural network framework for dynamic multi-valued mapping estimation and its applications Geng Li Di Qiu L. Lui 27 0 0 29 Jun 2024
Detecting Brittle Decisions for Free: Leveraging Margin Consistency in Deep Robust Classifiers Jonas Ngnawé Sabyasachi Sahoo Y. Pequignot Frédéric Precioso Christian Gagné AAML 42 0 0 26 Jun 2024
FedDr+: Stabilizing Dot-regression with Global Feature Distillation for Federated Learning Seongyoon Kim Minchan Jeong Sungnyun Kim Sungwoo Cho Sumyeong Ahn Se-Young Yun FedML 47 0 0 04 Jun 2024
Classifying Overlapping Gaussian Mixtures in High Dimensions: From Optimal Classifiers to Neural Nets Khen Cohen Noam Levi Yaron Oz BDL 31 1 0 28 May 2024
Linguistic Collapse: Neural Collapse in (Large) Language Models Robert Wu Vardan Papyan 48 12 0 28 May 2024
Pursuing Feature Separation based on Neural Collapse for Out-of-Distribution Detection Yingwen Wu Ruiji Yu Xinwen Cheng Zhengbao He Xiaolin Huang OODD 72 1 0 28 May 2024
Spectral regularization for adversarially-robust representation learning Sheng Yang Jacob A. Zavatone-Veth Cengiz Pehlevan AAML OOD 49 0 0 27 May 2024
WeiPer: OOD Detection using Weight Perturbations of Class Projections Maximilian Granz Manuel Heurich Tim Landgraf OODD 46 1 0 27 May 2024
Hamiltonian Mechanics of Feature Learning: Bottleneck Structure in Leaky ResNets Arthur Jacot Alexandre Kaiser 36 0 0 27 May 2024
Supervised Contrastive Representation Learning: Landscape Analysis with Unconstrained Features Tina Behnia Christos Thrampoulidis SSL 39 0 0 29 Feb 2024
Which Frequencies do CNNs Need? Emergent Bottleneck Structure in Feature Learning Yuxiao Wen Arthur Jacot 60 6 0 12 Feb 2024
Pushing Boundaries: Mixup's Influence on Neural Collapse Quinn Fisher Haoming Meng Vardan Papyan AAML UQCV 44 5 0 09 Feb 2024
Cross Entropy versus Label Smoothing: A Neural Collapse Perspective Li Guo Keith Ross Zifan Zhao George Andriopoulos Shuyang Ling Yufeng Xu Zixuan Dong UQCV NoLa 30 9 0 06 Feb 2024
Bias Mitigating Few-Shot Class-Incremental Learning Li-Jun Zhao Zhen-Duo Chen Ziya Zhang Xin Luo Xin-Shun Xu CLL 39 1 0 01 Feb 2024
Manipulating Sparse Double Descent Ya Shi Zhang 24 0 0 19 Jan 2024
Combating Representation Learning Disparity with Geometric Harmonization Zhihan Zhou Jiangchao Yao Feng Hong Ya Zhang Bo Han Yanfeng Wang 33 12 0 26 Oct 2023
Neural Collapse in Multi-label Learning with Pick-all-label Loss Pengyu Li Xiao Li Yutong Wang Qing Qu 30 8 0 24 Oct 2023
A Unified Approach to Domain Incremental Learning with Memory: Theory and Algorithm Haizhou Shi Hao Wang CLL 34 18 0 18 Oct 2023
Towards Demystifying the Generalization Behaviors When Neural Collapse Emerges Peifeng Gao Qianqian Xu Yibo Yang Peisong Wen Huiyang Shao Zhiyong Yang Guohao Li Qingming Huang AAML 28 3 0 12 Oct 2023
Generalized Neural Collapse for a Large Number of Classes Jiachen Jiang Jinxin Zhou Peng Wang Qing Qu Dustin Mixon Chong You Zhihui Zhu AI4CE 32 23 0 09 Oct 2023