On Feature Decorrelation in Self-Supervised Learning

2 May 2021

Yue Wang

Hang Zhao

SSL

OOD

ArXiv PDF HTML

Papers citing "On Feature Decorrelation in Self-Supervised Learning"

50 / 126 papers shown

Title
Rethinking Self-Supervised Learning Within the Framework of Partial Information Decomposition S. Mohamadi Gianfranco Doretto Donald Adjeroh SSL 71 0 0 03 Dec 2024
Direct Coloring for Self-Supervised Enhanced Feature Decoupling S. Mohamadi Gianfranco Doretto Donald Adjeroh 71 0 0 03 Dec 2024
Preventing Dimensional Collapse in Self-Supervised Learning via Orthogonality Regularization Junlin He Jinxiao Du Wei Ma SSL 35 0 0 01 Nov 2024
Rethinking Positive Pairs in Contrastive Learning Jiantao Wu Shentong Mo Zhenhua Feng Sara Atito Josef Kitler Muhammad Awais SSL VLM 46 3 0 23 Oct 2024
Generalized Probabilistic Attention Mechanism in Transformers DongNyeong Heo Heeyoul Choi 54 0 0 21 Oct 2024
Decorrelation-based Self-Supervised Visual Representation Learning for Writer Identification Arkadip Maitra Shree Mitra Siladittya Manna Saumik Bhattacharya Umapada Pal SSL 25 0 0 02 Oct 2024
Towards Automatic Assessment of Self-Supervised Speech Models using Rank Zakaria Aldeneh Vimal Thilak Takuya Higuchi B. Theobald Tatiana Likhomanenko SSL 75 0 0 16 Sep 2024
Exploiting Supervised Poison Vulnerability to Strengthen Self-Supervised Defense Jeremy A. Styborski Mingzhi Lyu Y. Huang Adams Kong 39 0 0 13 Sep 2024
Label-free Monitoring of Self-Supervised Learning Progress Isaac Xu Scott Lowe Thomas Trappenberg 32 1 0 10 Sep 2024
Whitening Consistently Improves Self-Supervised Learning András Kalapos Bálint Gyires-Tóth SSL 39 0 0 14 Aug 2024
On Learning Discriminative Features from Synthesized Data for Self-Supervised Fine-Grained Visual Recognition Zihu Wang Lingqiao Liu Scott Ricardo Figueroa Weston Samuel Tian Peng Li 29 2 0 19 Jul 2024
RealViformer: Investigating Attention for Real-World Video Super-Resolution Yuehan Zhang Angela Yao SupR 28 2 0 19 Jul 2024
Encourage or Inhibit Monosemanticity? Revisit Monosemanticity from a Feature Decorrelation Perspective Hanqi Yan Yanzheng Xiang Guangyi Chen Yifei Wang Lin Gui Yulan He 39 5 0 25 Jun 2024
SimSMoE: Solving Representational Collapse via Similarity Measure Giang Do Hung Le T. Tran MoE 41 1 0 22 Jun 2024
Words in Motion: Extracting Interpretable Control Vectors for Motion Transformers Ömer Sahin Tas Royden Wagner 51 1 0 17 Jun 2024
Towards Fundamentally Scalable Model Selection: Asymptotically Fast Update and Selection Wenxiao Wang Weiming Zhuang Lingjuan Lyu 41 0 0 11 Jun 2024
Federated Learning under Partially Class-Disjoint Data via Manifold Reshaping Ziqing Fan Jiangchao Yao Ruipeng Zhang Lingjuan Lyu Ya-Qin Zhang Yanfeng Wang FedML 32 2 0 29 May 2024
Bridging Mini-Batch and Asymptotic Analysis in Contrastive Learning: From InfoNCE to Kernel-Based Losses Panagiotis Koromilas Giorgos Bouritsas Theodoros Giannakopoulos M. Nicolaou Yannis Panagakis 32 0 0 28 May 2024
Capsule Network Projectors are Equivariant and Invariant Learners Miles Everett A. Durrant Mingjun Zhong Georgios Leontidis SSL 3DPC 45 2 0 23 May 2024
Unveiling Key Aspects of Fine-Tuning in Sentence Embeddings: A Representation Rank Analysis Euna Jung Jaeill Kim Jungmin Ko Jinwoo Park Wonjong Rhee 50 0 0 18 May 2024
Modeling Caption Diversity in Contrastive Vision-Language Pretraining Samuel Lavoie Polina Kirichenko Mark Ibrahim Mahmoud Assran Andrew Gordon Wilson Aaron Courville Nicolas Ballas CLIP VLM 64 19 0 30 Apr 2024
Hypergraph Self-supervised Learning with Sampling-efficient Signals Fan Li Xiaoyang Wang Dawei Cheng Wenjie Zhang Ying Zhang Xuemin Lin SSL 40 3 0 18 Apr 2024
AZ-NAS: Assembling Zero-Cost Proxies for Network Architecture Search Junghyup Lee Bumsub Ham 24 6 0 28 Mar 2024
Rethinking the Representation in Federated Unsupervised Learning with Non-IID Data Xinting Liao Weiming Liu Chaochao Chen Pengyang Zhou Fengyuan Yu Huabin Zhu Binhui Yao Tao Wang Xiaolin Zheng Yanchao Tan FedML 43 7 0 25 Mar 2024
Total Disentanglement of Font Images into Style and Character Class Features Daichi Haraguchi Wataru Shimoda Kota Yamaguchi Seiichi Uchida DRL 32 0 0 19 Mar 2024
Rethinking The Uniformity Metric in Self-Supervised Learning Xianghong Fang Jian Li Qiang Sun Benyou Wang SSL 12 0 0 01 Mar 2024
Ads Recommendation in a Collapsed and Entangled World Junwei Pan Wei Xue Ximei Wang Haibin Yu Xun Liu Shijie Quan Xueming Qiu Dapeng Liu Lei Xiao Jie Jiang 45 5 0 22 Feb 2024
Overcoming Dimensional Collapse in Self-supervised Contrastive Learning for Medical Image Segmentation Jamshid Hassanpour V. Srivastav Didier Mutter N. Padoy SSL 47 2 0 22 Feb 2024
Unsupervised Concept Discovery Mitigates Spurious Correlations Md Rifat Arefin Yan Zhang A. Baratin Francesco Locatello Irina Rish Dianbo Liu Kenji Kawaguchi 54 6 0 20 Feb 2024
CCFC++: Enhancing Federated Clustering through Feature Decorrelation Jie Yan Jing Liu Yi-Zi Ning Zhonghan Zhang FedML 19 2 0 20 Feb 2024
Scalable Graph Self-Supervised Learning Ali Saheb Pasand Reza Moravej Mahdi Biparva Raika Karimi Ali Ghodsi SSL 11 0 0 14 Feb 2024
WERank: Towards Rank Degradation Prevention for Self-Supervised Learning Using Weight Regularization Ali Saheb Pasand Reza Moravej Mahdi Biparva Ali Ghodsi 34 2 0 14 Feb 2024
Prototypical Contrastive Learning through Alignment and Uniformity for Recommendation Yangxun Ou Lei Chen Fenglin Pan Yupeng Wu 19 1 0 03 Feb 2024
Memorization in Self-Supervised Learning Improves Downstream Generalization Wenhao Wang Muhammad Ahmad Kaleem Adam Dziedzic Michael Backes Nicolas Papernot Franziska Boenisch SSL 17 9 0 19 Jan 2024
LDReg: Local Dimensionality Regularized Self-Supervised Learning Hanxun Huang R. Campello S. Erfani Xingjun Ma Michael E. Houle James Bailey 30 5 0 19 Jan 2024
Enhancing Contrastive Learning with Efficient Combinatorial Positive Pairing Jaeill Kim Duhun Hwang Eunjung Lee Jangwon Suh Jimyeong Kim Wonjong Rhee 28 0 0 11 Jan 2024
Self-supervised learning for skin cancer diagnosis with limited training data Hamish Haggerty Rohitash Chandra 23 3 0 01 Jan 2024
3DTINC: Time-Equivariant Non-Contrastive Learning for Predicting Disease Progression from Longitudinal OCTs T. Emre A. Chakravarty Antoine Rivail Dmitrii Lachinov Oliver Leingang ... S. Sivaprasad Daniel Rueckert A. Lotery U. Schmidt-Erfurth Hrvoje Bogunović MedIm 27 3 0 28 Dec 2023
LiDAR: Sensing Linear Probing Performance in Joint Embedding SSL Architectures Vimal Thilak Chen Huang Omid Saremi Laurent Dinh Hanlin Goh Preetum Nakkiran Josh Susskind Etai Littwin 18 7 0 07 Dec 2023
Improving Self-supervised Molecular Representation Learning using Persistent Homology Yuankai Luo Lei Shi Veronika Thost SSL 29 8 0 29 Nov 2023
Architecture Matters: Uncovering Implicit Mechanisms in Graph Contrastive Learning Xiaojun Guo Yifei Wang Zeming Wei Yisen Wang 31 11 0 05 Nov 2023
Cross-Level Distillation and Feature Denoising for Cross-Domain Few-Shot Classification Hao Zheng Runqi Wang Jianzhuang Liu Asako Kanezaki 30 7 0 04 Nov 2023
Gramian Attention Heads are Strong yet Efficient Vision Learners Jongbin Ryu Dongyoon Han J. Lim 30 1 0 25 Oct 2023
On the Embedding Collapse when Scaling up Recommendation Models Xingzhuo Guo Kai Yan Ximei Wang Baixu Chen Haohan Wang Mingsheng Long 13 21 0 06 Oct 2023
DimCL: Dimensional Contrastive Learning For Improving Self-Supervised Learning Thanh Nguyen T. Pham Chaoning Zhang T. Luu Thang Vu Chang-Dong Yoo 25 9 0 21 Sep 2023
Layer-wise training for self-supervised learning on graphs Oscar Pina Verónica Vilaplana SSL GNN 30 1 0 04 Sep 2023
Adaptive whitening with fast gain modulation and slow synaptic plasticity Lyndon Duong Eero P. Simoncelli D. Chklovskii David Lipshutz 24 4 0 25 Aug 2023
Feature-Suppressed Contrast for Self-Supervised Food Pre-training Xinda Liu Yaohui Zhu Linhu Liu Jiang Tian Lili Wang SSL 20 2 0 07 Aug 2023
Stochastic positional embeddings improve masked image modeling Amir Bar Florian Bordes Assaf Shocher Mahmoud Assran Pascal Vincent Nicolas Ballas Trevor Darrell Amir Globerson Yann LeCun 18 3 0 31 Jul 2023
Towards the Sparseness of Projection Head in Self-Supervised Learning Zeen Song Xingzhe Su Jingyao Wang Wenwen Qiang Changwen Zheng Fuchun Sun 33 3 0 18 Jul 2023