Towards Demystifying Representation Learning with Non-contrastive Self-supervision

11 October 2021

Papers citing "Towards Demystifying Representation Learning with Non-contrastive Self-supervision"

23 / 23 papers shown

Title
Representation Learning via Non-Contrastive Mutual Information Z. Guo Bernardo Avila-Pires Khimya Khetarpal Dale Schuurmans Bo Dai SSL 61 0 0 23 Apr 2025
Exploiting Supervised Poison Vulnerability to Strengthen Self-Supervised Defense Jeremy A. Styborski Mingzhi Lyu Y. Huang Adams Kong 39 0 0 13 Sep 2024
Investigating the Benefits of Projection Head for Representation Learning Yihao Xue Eric Gan Jiayi Ni Siddharth Joshi Baharan Mirzasoleiman 41 10 0 18 Mar 2024
$V_kD:$ Improving Knowledge Distillation using Orthogonal Projections Roy Miles Ismail Elezi Jiankang Deng 44 10 0 10 Mar 2024
Feature Normalization Prevents Collapse of Non-contrastive Learning Dynamics Han Bao SSL MLT 29 1 0 28 Sep 2023
A Spatiotemporal Correspondence Approach to Unsupervised LiDAR Segmentation with Traffic Applications Xiao Li Pan He Aotian Wu Sanjay Ranka Anand Rangarajan 3DPC 16 1 0 23 Aug 2023
A Cookbook of Self-Supervised Learning Randall Balestriero Mark Ibrahim Vlad Sobal Ari S. Morcos Shashank Shekhar ... Pierre Fernandez Amir Bar Hamed Pirsiavash Yann LeCun Micah Goldblum SyDa FedML SSL 44 273 0 24 Apr 2023
Rethinking Evaluation Protocols of Visual Representations Learned via Self-supervised Learning Jaehoon Lee Doyoung Yoon Byeongmoon Ji Kyungyul Kim Sangheum Hwang SSL 55 3 0 07 Apr 2023
Random Teachers are Good Teachers Felix Sarnthein Gregor Bachmann Sotiris Anagnostidis Thomas Hofmann 21 4 0 23 Feb 2023
The Edge of Orthogonality: A Simple View of What Makes BYOL Tick Pierre Harvey Richemond Allison C. Tam Yunhao Tang Florian Strub Bilal Piot Felix Hill SSL 28 9 0 09 Feb 2023
SemPPL: Predicting pseudo-labels for better contrastive representations Matko Bovsnjak Pierre Harvey Richemond Nenad Tomašev Florian Strub Jacob Walker Felix Hill Lars Buesing Razvan Pascanu Charles Blundell Jovana Mitrović SSL VLM 41 9 0 12 Jan 2023
Implicit variance regularization in non-contrastive SSL Manu Srinath Halvagal Axel Laborieux Friedemann Zenke 36 8 0 09 Dec 2022
Understanding Self-Predictive Learning for Reinforcement Learning Yunhao Tang Z. Guo Pierre Harvey Richemond Bernardo Avila-Pires Yash Chandak ... S. Thakoor Will Dabney Bilal Piot Daniele Calandriello Michal Valko SSL 27 28 0 06 Dec 2022
Understanding the Role of Nonlinearity in Training Dynamics of Contrastive Learning Yuandong Tian MLT 18 13 0 02 Jun 2022
Contrastive and Non-Contrastive Self-Supervised Learning Recover Global and Local Spectral Embedding Methods Randall Balestriero Yann LeCun SSL 18 129 0 23 May 2022
The Mechanism of Prediction Head in Non-contrastive Self-supervised Learning Zixin Wen Yuanzhi Li SSL 27 34 0 12 May 2022
Contrasting the landscape of contrastive and non-contrastive learning Ashwini Pokle Jinjin Tian Yuchen Li Andrej Risteski SSL 30 28 0 29 Mar 2022
One Network Doesn't Rule Them All: Moving Beyond Handcrafted Architectures in Self-Supervised Learning Sharath Girish Debadeepta Dey Neel Joshi Vibhav Vineet S. Shah C. C. T. Mendes Abhinav Shrivastava Yale Song SSL 40 2 0 15 Mar 2022
The Power of Contrast for Feature Learning: A Theoretical Analysis Wenlong Ji Zhun Deng Ryumei Nakada James Y. Zou Linjun Zhang SSL 51 48 0 06 Oct 2021
Learning distinct features helps, provably Firas Laakom Jenni Raitoharju Alexandros Iosifidis M. Gabbouj MLT 30 6 0 10 Jun 2021
On Feature Decorrelation in Self-Supervised Learning Tianyu Hua Wenxiao Wang Zihui Xue Sucheng Ren Yue Wang Hang Zhao SSL OOD 124 187 0 02 May 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 317 5,775 0 29 Apr 2021
Understanding self-supervised Learning Dynamics without Contrastive Pairs Yuandong Tian Xinlei Chen Surya Ganguli SSL 138 279 0 12 Feb 2021