Prevalence of Neural Collapse during the terminal phase of deep learning training

18 August 2020

Papers citing "Prevalence of Neural Collapse during the terminal phase of deep learning training"

50 / 130 papers shown

Title
Generalized Neural Collapse for a Large Number of Classes Jiachen Jiang Jinxin Zhou Peng Wang Qing Qu Dustin Mixon Chong You Zhihui Zhu AI4CE 34 24 0 09 Oct 2023
Spectral alignment of stochastic gradient descent for high-dimensional classification tasks Gerard Ben Arous Reza Gheissari Jiaoyang Huang Aukosh Jagannath 35 14 0 04 Oct 2023
Fundamental Limits of Deep Learning-Based Binary Classifiers Trained with Hinge Loss T. Getu Georges Kaddoum M. Bennis 40 1 0 13 Sep 2023
SegDA: Maximum Separable Segment Mask with Pseudo Labels for Domain Adaptive Semantic Segmentation Anant Khandelwal 24 1 0 10 Aug 2023
Novel Class Discovery for Long-tailed Recognition Zhang Chuyu Rui Xu Xuming He 35 16 0 06 Aug 2023
Neural Collapse Terminus: A Unified Solution for Class Incremental Learning and Its Variants Yibo Yang Haobo Yuan Hefei Ling Jianlong Wu Lefei Zhang Zhouchen Lin Philip Torr Dacheng Tao Guohao Li CLL 31 8 0 03 Aug 2023
Get the Best of Both Worlds: Improving Accuracy and Transferability by Grassmann Class Representation Haoqi Wang Zhizhong Li Wayne Zhang 23 2 0 03 Aug 2023
A Neural Collapse Perspective on Feature Evolution in Graph Neural Networks Vignesh Kothapalli Tom Tirer Joan Bruna 38 11 0 04 Jul 2023
Precise Asymptotic Generalization for Multiclass Classification with Overparameterized Linear Models David X. Wu A. Sahai 26 2 0 23 Jun 2023
Symmetric Neural-Collapse Representations with Supervised Contrastive Loss: The Impact of ReLU and Batching Ganesh Ramachandra Kini V. Vakilian Tina Behnia Jaidev Gill Christos Thrampoulidis 17 1 0 13 Jun 2023
Quantifying the Variability Collapse of Neural Networks Jing-Xue Xu Haoxiong Liu 36 5 0 06 Jun 2023
Deep neural networks architectures from the perspective of manifold learning German Magai AAML AI4CE 24 6 0 06 Jun 2023
Unraveling Projection Heads in Contrastive Learning: Insights from Expansion and Shrinkage Yu Gui Cong Ma Yiqiao Zhong 22 7 0 06 Jun 2023
Reward Collapse in Aligning Large Language Models Ziang Song Tianle Cai Jason D. Lee Weijie J. Su ALM 33 22 0 28 May 2023
Reverse Engineering Self-Supervised Learning Ido Ben-Shaul Ravid Shwartz-Ziv Tomer Galanti S. Dekel Yann LeCun SSL 26 34 0 24 May 2023
Mitigating Label Noise through Data Ambiguation Julian Lienen Eyke Hüllermeier NoLa 32 7 0 23 May 2023
Towards understanding neural collapse in supervised contrastive learning with the information bottleneck method Siwei Wang S. Palmer 32 2 0 19 May 2023
The emergence of clusters in self-attention dynamics Borjan Geshkovski Cyril Letrouit Yury Polyanskiy Philippe Rigollet 22 46 0 09 May 2023
Neural Collapse Inspired Federated Learning with Non-iid Data Chenxi Huang Liang Xie Yibo Yang Wenxiao Wang Binbin Lin D. Cai FedML 34 3 0 27 Mar 2023
Type-II Saddles and Probabilistic Stability of Stochastic Gradient Descent Liu Ziyin Botao Li Tomer Galanti Masakuni Ueda 37 7 0 23 Mar 2023
No Fear of Classifier Biases: Neural Collapse Inspired Federated Learning with Synthetic and Fixed Classifier Zexi Li Xinyi Shang Rui He Tao R. Lin Chao Wu FedML 47 54 0 17 Mar 2023
Generalizing and Decoupling Neural Collapse via Hyperspherical Uniformity Gap Weiyang Liu L. Yu Adrian Weller Bernhard Schölkopf 37 18 0 11 Mar 2023
Inducing Neural Collapse to a Fixed Hierarchy-Aware Frame for Reducing Mistake Severity Tong Liang Jim Davis 35 10 0 10 Mar 2023
Cut your Losses with Squentropy Like Hui M. Belkin S. Wright UQCV 18 8 0 08 Feb 2023
Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class Incremental Learning Yibo Yang Haobo Yuan Hefei Ling Zhouchen Lin Philip Torr Dacheng Tao CLL 34 98 0 06 Feb 2023
Learning Prototype Classifiers for Long-Tailed Recognition Saurabh Sharma Yongqin Xian Ning Yu Ambuj K. Singh 37 13 0 01 Feb 2023
Unsupervised Manifold Linearizing and Clustering Tianjiao Ding Shengbang Tong Kwan Ho Ryan Chan Xili Dai Y. Ma B. Haeffele 25 11 0 04 Jan 2023
Understanding Imbalanced Semantic Segmentation Through Neural Collapse Zhisheng Zhong Jiequan Cui Yibo Yang Xiaoyang Wu Xiaojuan Qi Xinming Zhang Jiaya Jia 138 45 0 03 Jan 2023
Problem-Dependent Power of Quantum Neural Networks on Multi-Class Classification Yuxuan Du Yibo Yang Dacheng Tao Min-hsiu Hsieh 41 23 0 29 Dec 2022
Principled and Efficient Transfer Learning of Deep Models via Neural Collapse Xiao Li Sheng Liu Jin-li Zhou Xin Lu C. Fernandez‐Granda Zhihui Zhu Q. Qu AAML 28 19 0 23 Dec 2022
Learning useful representations for shifting tasks and distributions Jianyu Zhang Léon Bottou OOD 34 13 0 14 Dec 2022
CRAFT: Concept Recursive Activation FacTorization for Explainability Thomas Fel Agustin Picard Louis Bethune Thibaut Boissin David Vigouroux Julien Colin Rémi Cadène Thomas Serre 19 103 0 17 Nov 2022
A Law of Data Separation in Deep Learning Hangfeng He Weijie J. Su OOD 24 37 0 31 Oct 2022
Perturbation Analysis of Neural Collapse Tom Tirer Haoxiang Huang Jonathan Niles-Weed AAML 40 24 0 29 Oct 2022
Class Based Thresholding in Early Exit Semantic Segmentation Networks Alperen Görmez Erdem Koyuncu 23 5 0 27 Oct 2022
Grokking phase transitions in learning local rules with gradient descent Bojan Žunkovič E. Ilievski 63 16 0 26 Oct 2022
Hidden State Variability of Pretrained Language Models Can Guide Computation Reduction for Transfer Learning Shuo Xie Jiahao Qiu Ankita Pasad Li Du Qing Qu Hongyuan Mei 35 16 0 18 Oct 2022
Continual Learning by Modeling Intra-Class Variation L. Yu Tianyang Hu Lanqing Hong Zhen Liu Adrian Weller Weiyang Liu CLL 48 12 0 11 Oct 2022
Multi-CLS BERT: An Efficient Alternative to Traditional Ensembling Haw-Shiuan Chang Ruei-Yao Sun Kathryn Ricci Andrew McCallum 43 14 0 10 Oct 2022
Class-wise and reduced calibration methods Michael Panchenko Anes Benmerzoug Miguel de Benito Delgado 21 0 0 07 Oct 2022
Are All Losses Created Equal: A Neural Collapse Perspective Jinxin Zhou Chong You Xiao Li Kangning Liu Sheng Liu Qing Qu Zhihui Zhu 36 59 0 04 Oct 2022
What shapes the loss landscape of self-supervised learning? Liu Ziyin Ekdeep Singh Lubana Masakuni Ueda Hidenori Tanaka 50 20 0 02 Oct 2022
Neural Collapse with Normalized Features: A Geometric Analysis over the Riemannian Manifold Can Yaras Peng Wang Zhihui Zhu Laura Balzano Qing Qu 25 42 0 19 Sep 2022
Importance Tempering: Group Robustness for Overparameterized Models Yiping Lu Wenlong Ji Zachary Izzo Lexing Ying 42 7 0 19 Sep 2022
Improving Self-Supervised Learning by Characterizing Idealized Representations Yann Dubois Tatsunori Hashimoto Stefano Ermon Percy Liang SSL 83 40 0 13 Sep 2022
DOMINO: Domain-aware Model Calibration in Medical Image Segmentation Skylar E. Stolte Kyle Volle A. Indahlastari Alejandro Albizu A. Woods Kevin Brink Matthew Hale R. Fang 25 5 0 13 Sep 2022
Imbalance Trouble: Revisiting Neural-Collapse Geometry Christos Thrampoulidis Ganesh Ramachandra Kini V. Vakilian Tina Behnia 26 69 0 10 Aug 2022
Balanced Contrastive Learning for Long-Tailed Visual Recognition Jianggang Zhu Zihan Wang Jingjing Chen Yi-Ping Phoebe Chen Yueping Jiang 24 167 0 19 Jul 2022
Learning sparse features can lead to overfitting in neural networks Leonardo Petrini Francesco Cagnetta Eric Vanden-Eijnden M. Wyart MLT 42 23 0 24 Jun 2022
DECK: Model Hardening for Defending Pervasive Backdoors Guanhong Tao Yingqi Liu Shuyang Cheng Shengwei An Zhuo Zhang Qiuling Xu Guangyu Shen Xiangyu Zhang AAML 26 7 0 18 Jun 2022