MaxSup: Overcoming Representation Collapse in Label Smoothing

18 February 2025

Papers citing "MaxSup: Overcoming Representation Collapse in Label Smoothing"

32 / 32 papers shown

Title
The Persistence of Neural Collapse Despite Low-Rank Bias: An Analytic Perspective Through Unconstrained Features Connall Garrod Jonathan P. Keating 53 4 0 30 Oct 2024
Neural Collapse versus Low-rank Bias: Is Deep Neural Collapse Really Optimal? Peter Súkeník Marco Mondelli Christoph H. Lampert AI4CE 58 10 0 23 May 2024
Cross Entropy versus Label Smoothing: A Neural Collapse Perspective Li Guo George Andriopoulos Zifan Zhao Shuyang Ling Shuyang Ling Keith Ross UQCV NoLa 58 9 0 06 Feb 2024
Steering Large Language Models for Machine Translation with Finetuning and In-Context Learning Duarte M. Alves Nuno M. Guerreiro Joao Alves José P. Pombal Ricardo Rei José G. C. de Souza Pierre Colombo André F.T. Martins 63 52 0 20 Oct 2023
Quantifying the Variability Collapse of Neural Networks Jing-Xue Xu Haoxiong Liu 62 6 0 06 Jun 2023
Rethinking Confidence Calibration for Failure Prediction Fei Zhu Zhen Cheng Xu-Yao Zhang Cheng-Lin Liu UQCV 75 41 0 06 Mar 2023
CHiLS: Zero-Shot Image Classification with Hierarchical Label Sets Zachary Novack Julian McAuley Zachary Chase Lipton Saurabh Garg VLM 77 83 0 06 Feb 2023
Adaptive Label Smoothing with Self-Knowledge in Natural Language Generation Dongkyu Lee Ka Chun Cheung N. Zhang 41 8 0 22 Oct 2022
Are All Losses Created Equal: A Neural Collapse Perspective Jinxin Zhou Chong You Xiao Li Kangning Liu Sheng Liu Qing Qu Zhihui Zhu 64 64 0 04 Oct 2022
Efficient One Pass Self-distillation with Zipf's Label Smoothing Jiajun Liang Linze Li Z. Bing Borui Zhao Yao Tang Bo Lin Haoqiang Fan 47 19 0 26 Jul 2022
No Reason for No Supervision: Improved Generalization in Supervised Models Mert Bulent Sariyildiz Yannis Kalantidis Alahari Karteek Diane Larlus SSL OOD LRM 65 28 0 30 Jun 2022
Revisiting Label Smoothing and Knowledge Distillation Compatibility: What was Missing? Keshigeyan Chandrasegaran Ngoc-Trung Tran Yunqing Zhao Ngai-Man Cheung 120 44 0 29 Jun 2022
SP-ViT: Learning 2D Spatial Priors for Vision Transformers Yuxuan Zhou Wangmeng Xiang Chong Li Biao Wang Xihan Wei Lei Zhang Margret Keuper Xia Hua ViT 57 15 0 15 Jun 2022
Exploring Hierarchical Graph Representation for Large-Scale Zero-Shot Image Classification Kai Yi Xiaoqian Shen Yunhao Gou Mohamed Elhoseiny 72 21 0 02 Mar 2022
Rethinking Supervised Pre-training for Better Downstream Transferring Yutong Feng Jianwen Jiang Mingqian Tang Rong Jin Yue Gao SSL 88 40 0 12 Oct 2021
HSVA: Hierarchical Semantic-Visual Adaptation for Zero-Shot Learning Shiming Chen Guosen Xie Yang Liu Qinmu Peng Baigui Sun Hao Li Xinge You Ling Shao 102 128 0 30 Sep 2021
Is Label Smoothing Truly Incompatible with Knowledge Distillation: An Empirical Study Zhiqiang Shen Zechun Liu Dejia Xu Zitian Chen Kwang-Ting Cheng Marios Savvides 41 76 0 01 Apr 2021
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows Ze Liu Yutong Lin Yue Cao Han Hu Yixuan Wei Zheng Zhang Stephen Lin B. Guo ViT 415 21,347 0 25 Mar 2021
Training data-efficient image transformers & distillation through attention Hugo Touvron Matthieu Cord Matthijs Douze Francisco Massa Alexandre Sablayrolles Hervé Jégou ViT 357 6,731 0 23 Dec 2020
Delving Deep into Label Smoothing Chang-Bin Zhang Peng-Tao Jiang Qibin Hou Yunchao Wei Qi Han Zhen Li Ming-Ming Cheng 96 212 0 25 Nov 2020
Why Do Better Loss Functions Lead to Less Transferable Features? Simon Kornblith Ting-Li Chen Honglak Lee Mohammad Norouzi FaML 75 92 0 30 Oct 2020
When Does Label Smoothing Help? Rafael Müller Simon Kornblith Geoffrey E. Hinton UQCV 179 1,938 0 06 Jun 2019
Unified Perceptual Parsing for Scene Understanding Tete Xiao Yingcheng Liu Bolei Zhou Yuning Jiang Jian Sun OCL VOS 171 1,871 0 26 Jul 2018
MobileNetV2: Inverted Residuals and Linear Bottlenecks Mark Sandler Andrew G. Howard Menglong Zhu A. Zhmoginov Liang-Chieh Chen 171 19,204 0 13 Jan 2018
On Calibration of Modern Neural Networks Chuan Guo Geoff Pleiss Yu Sun Kilian Q. Weinberger UQCV 280 5,812 0 14 Jun 2017
Regularizing Neural Networks by Penalizing Confident Output Distributions Gabriel Pereyra George Tucker J. Chorowski Lukasz Kaiser Geoffrey E. Hinton NoLa 155 1,137 0 23 Jan 2017
Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization Ramprasaath R. Selvaraju Michael Cogswell Abhishek Das Ramakrishna Vedantam Devi Parikh Dhruv Batra FAtt 264 19,929 0 07 Oct 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.1K 193,426 0 10 Dec 2015
Rethinking the Inception Architecture for Computer Vision Christian Szegedy Vincent Vanhoucke Sergey Ioffe Jonathon Shlens Z. Wojna 3DV BDL 804 27,303 0 02 Dec 2015
Distilling the Knowledge in a Neural Network Geoffrey E. Hinton Oriol Vinyals J. Dean FedML 320 19,609 0 09 Mar 2015
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 1.6K 39,472 0 01 Sep 2014
Visualizing and Understanding Convolutional Networks Matthew D. Zeiler Rob Fergus FAtt SSL 541 15,874 0 12 Nov 2013