On Feature Decorrelation in Self-Supervised Learning

2 May 2021

Yue Wang

Hang Zhao

SSL

OOD

ArXiv PDF HTML

Papers citing "On Feature Decorrelation in Self-Supervised Learning"

50 / 126 papers shown

Title
More Synergy, Less Redundancy: Exploiting Joint Mutual Information for Self-Supervised Learning S. Mohamadi Gianfranco Doretto Don Adjeroh SSL 10 9 0 02 Jul 2023
On the Importance of Feature Decorrelation for Unsupervised Representation Learning in Reinforcement Learning Hojoon Lee Ko-tik Lee Dongyoon Hwang Hyunho Lee ByungKun Lee Jaegul Choo SSL OOD 26 5 0 09 Jun 2023
Unraveling Projection Heads in Contrastive Learning: Insights from Expansion and Shrinkage Yu Gui Cong Ma Yiqiao Zhong 22 6 0 06 Jun 2023
Feature Learning in Image Hierarchies using Functional Maximal Correlation Bo Hu Yuheng Bu José C. Príncipe 19 0 0 31 May 2023
Whitening-based Contrastive Learning of Sentence Embeddings Wenjie Zhuo Yifan Sun Xiaohan Wang Linchao Zhu Yezhou Yang 30 18 0 28 May 2023
Modulate Your Spectrum in Self-Supervised Learning Xi Weng Yu-Li Ni Tengwei Song Jie Luo Rao Muhammad Anwer Salman Khan F. Khan Lei Huang 32 5 0 26 May 2023
Unsupervised Embedding Quality Evaluation Anton Tsitsulin Marina Munkhoeva Bryan Perozzi SSL 6 6 0 26 May 2023
Sharpness-Aware Minimization Leads to Low-Rank Features Maksym Andriushchenko Dara Bahri H. Mobahi Nicolas Flammarion AAML 25 25 0 25 May 2023
Know Your Self-supervised Learning: A Survey on Image-based Generative and Discriminative Training Utku Ozbulak Hyun Jung Lee Beril Boga Esla Timothy Anzaku Ho-min Park Arnout Van Messem W. D. Neve J. Vankerschaver DiffM 26 36 0 23 May 2023
Learning Semi-supervised Gaussian Mixture Models for Generalized Category Discovery Bingchen Zhao Xin Wen Kai Han 33 46 0 10 May 2023
Self-supervised dense representation learning for live-cell microscopy with time arrow prediction Benjamin Gallusser Max Stieber Martin Weigert 13 7 0 09 May 2023
A Cookbook of Self-Supervised Learning Randall Balestriero Mark Ibrahim Vlad Sobal Ari S. Morcos Shashank Shekhar ... Pierre Fernandez Amir Bar Hamed Pirsiavash Yann LeCun Micah Goldblum SyDa FedML SSL 44 273 0 24 Apr 2023
VNE: An Effective Method for Improving Deep Representation by Manipulating Eigenvalue Distribution Jaeill Kim Suhyun Kang Duhun Hwang Jungwook Shin Wonjong Rhee DRL 13 21 0 04 Apr 2023
Towards Understanding the Mechanism of Contrastive Learning via Similarity Structure: A Theoretical Analysis Hiroki Waida Yuichiro Wada Léo Andéol Takumi Nakagawa Yuhui Zhang Takafumi Kanamori SSL 23 5 0 01 Apr 2023
Feature propagation as self-supervision signals on graphs Oscar Pina Verónica Vilaplana SSL 12 1 0 15 Mar 2023
A Message Passing Perspective on Learning Dynamics of Contrastive Learning Yifei Wang Qi Zhang Tianqi Du Jiansheng Yang Zhouchen Lin Yisen Wang SSL 32 18 0 08 Mar 2023
Towards a Unified Theoretical Understanding of Non-contrastive Learning via Rank Differential Mechanism Zhijian Zhuo Yifei Wang Jinwen Ma Yisen Wang 45 24 0 04 Mar 2023
Can representation learning for multimodal image registration be improved by supervision of intermediate layers? Elisabeth Wetzer Joakim Lindblad Natavsa Sladoje SSL 10 1 0 01 Mar 2023
Evaluating Self-Supervised Learning via Risk Decomposition Yann Dubois Tatsunori Hashimoto Percy Liang 14 9 0 06 Feb 2023
Rethinking Semi-Supervised Medical Image Segmentation: A Variance-Reduction Perspective Chenyu You Weicheng Dai Yifei Min Fenglin Liu David A. Clifton S. Kevin Zhou Lawrence H. Staib James S Duncan 23 69 0 03 Feb 2023
OrthoReg: Improving Graph-regularized MLPs via Orthogonality Regularization Hengrui Zhang Shen Wang V. Ioannidis Soji Adeshina Jiani Zhang Xiao Qin Christos Faloutsos Da Zheng George Karypis Philip S. Yu 24 4 0 31 Jan 2023
Adaptive whitening in neural populations with gain-modulating interneurons Lyndon Duong David Lipshutz D. Heeger D. Chklovskii Eero P. Simoncelli 26 4 0 27 Jan 2023
Unifying Synergies between Self-supervised Learning and Dynamic Computation Tarun Krishna Ayush K. Rai Alexandru Drimbarean Eric Arazo Paul Albert Alan F. Smeaton Kevin McGuinness Noel E. O'Connor 24 0 0 22 Jan 2023
A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trends Jie Gui Tuo Chen Jing Zhang Qiong Cao Zhe Sun Haoran Luo Dacheng Tao 31 124 0 13 Jan 2023
Learning Decorrelated Representations Efficiently Using Fast Fourier Transform Yutaro Shigeto Masashi Shimbo Yuya Yoshikawa A. Takeuchi 19 0 0 04 Jan 2023
A Unified Framework for Contrastive Learning from a Perspective of Affinity Matrix Wenbin Li Meihao Kong Xuesong Yang Lei Wang Jing Huo Yang Gao Jiebo Luo 33 0 0 26 Nov 2022
Ladder Siamese Network: a Method and Insights for Multi-level Self-Supervised Learning Ryota Yoshihashi Shuhei Nishimura Dai Yonebayashi Yuya Otsuka Tomohiro Tanaka Takashi Miyazaki SSL 24 2 0 25 Nov 2022
EquiMod: An Equivariance Module to Improve Self-Supervised Learning Alexandre Devillers Mathieu Lefort SSL 13 18 0 02 Nov 2022
Augmentation Invariant Manifold Learning Shulei Wang 36 1 0 01 Nov 2022
The Curious Case of Benign Memorization Sotiris Anagnostidis Gregor Bachmann Lorenzo Noci Thomas Hofmann AAML 46 8 0 25 Oct 2022
Non-Contrastive Learning-based Behavioural Biometrics for Smart IoT Devices Oshan Jayawardena Fariza Rashid Suranga Seneviratne 11 0 0 24 Oct 2022
Self-Supervised Learning via Maximum Entropy Coding Xin Liu Zhongdao Wang Yali Li Shengjin Wang SSL 24 40 0 20 Oct 2022
Rethinking Prototypical Contrastive Learning through Alignment, Uniformity and Correlation Shentong Mo Zhun Sun Chao Li 15 11 0 18 Oct 2022
Learning Self-Regularized Adversarial Views for Self-Supervised Vision Transformers Tao Tang Changlin Li Guangrun Wang Kaicheng Yu Xiaojun Chang Xiaodan Liang ViT 28 1 0 16 Oct 2022
How Mask Matters: Towards Theoretical Understandings of Masked Autoencoders Qi Zhang Yifei Wang Yisen Wang 25 73 0 15 Oct 2022
Unsupervised Few-shot Learning via Deep Laplacian Eigenmaps Kuilin Chen Chi-Guhn Lee SSL 40 3 0 07 Oct 2022
RankMe: Assessing the downstream performance of pretrained self-supervised representations by their rank Q. Garrido Randall Balestriero Laurent Najman Yann LeCun SSL 50 72 0 05 Oct 2022
What shapes the loss landscape of self-supervised learning? Liu Ziyin Ekdeep Singh Lubana Masakuni Ueda Hidenori Tanaka 50 20 0 02 Oct 2022
Towards Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learning Yujun Shi Jian Liang Wenqing Zhang Vincent Y. F. Tan Song Bai FedML 79 55 0 01 Oct 2022
Understanding Collapse in Non-Contrastive Siamese Representation Learning Alexander C. Li Alexei A. Efros Deepak Pathak SSL 45 33 0 29 Sep 2022
Variance Covariance Regularization Enforces Pairwise Independence in Self-Supervised Representations Grégoire Mialon Randall Balestriero Yann LeCun 29 9 0 29 Sep 2022
Interneurons accelerate learning dynamics in recurrent neural networks for statistical adaptation David Lipshutz C. Pehlevan D. Chklovskii 25 11 0 21 Sep 2022
The Geometry of Self-supervised Learning Models and its Impact on Transfer Learning Romain Cosentino Sarath Shekkizhar Mahdi Soltanolkotabi A. Avestimehr Antonio Ortega SSL 51 7 0 18 Sep 2022
Self-Supervised Learning with an Information Maximization Criterion Serdar Ozsoy Shadi S. Hamdan Sercan Ö. Arik Deniz Yuret A. Erdogan SSL 21 35 0 16 Sep 2022
Improving Self-Supervised Learning by Characterizing Idealized Representations Yann Dubois Tatsunori Hashimoto Stefano Ermon Percy Liang SSL 83 40 0 13 Sep 2022
What can we Learn by Predicting Accuracy? Olivier Risser-Maroix Benjamin Chamand 27 4 0 02 Aug 2022
Revisiting the Critical Factors of Augmentation-Invariant Representation Learning Junqiang Huang Xiangwen Kong Xiangyu Zhang 30 6 0 30 Jul 2022
KG-NSF: Knowledge Graph Completion with a Negative-Sample-Free Approach A.B. Bahaj Safae Lhazmir Mounir Ghogho 11 1 0 29 Jul 2022
Consistency is the key to further mitigating catastrophic forgetting in continual learning Prashant Bhat Bahram Zonooz Elahe Arani CLL 32 16 0 11 Jul 2022
Evaluating Self-Supervised Learning for Molecular Graph Embeddings Hanchen Wang Jean Kaddour Shengchao Liu Jian Tang Joan Lasenby Qi Liu 27 20 0 16 Jun 2022