Implicit variance regularization in non-contrastive SSL

9 December 2022

Friedemann Zenke

Papers citing "Implicit variance regularization in non-contrastive SSL"

31 / 31 papers shown

Title
PhiNets: Brain-inspired Non-contrastive Learning Based on Temporal Prediction Hypothesis Satoki Ishikawa Makoto Yamada Han Bao Yuki Takezawa 118 0 0 23 May 2024
On the Stepwise Nature of Self-Supervised Learning James B. Simon Maksis Knutins Liu Ziyin Daniel Geisz Abraham J. Fetterman Joshua Albrecht SSL 55 34 0 27 Mar 2023
The Edge of Orthogonality: A Simple View of What Makes BYOL Tick Pierre Harvey Richemond Allison C. Tam Yunhao Tang Florian Strub Bilal Piot Felix Hill SSL 44 11 0 09 Feb 2023
Improving Self-Supervised Learning by Characterizing Idealized Representations Yann Dubois Tatsunori Hashimoto Stefano Ermon Percy Liang SSL 95 40 0 13 Sep 2022
On the duality between contrastive and non-contrastive self-supervised learning Q. Garrido Yubei Chen Adrien Bardes Laurent Najman Yann LeCun SSL 51 90 0 03 Jun 2022
Contrastive and Non-Contrastive Self-Supervised Learning Recover Global and Local Spectral Embedding Methods Randall Balestriero Yann LeCun SSL 73 132 0 23 May 2022
Masked Siamese Networks for Label-Efficient Learning Mahmoud Assran Mathilde Caron Ishan Misra Piotr Bojanowski Florian Bordes Pascal Vincent Armand Joulin Michael G. Rabbat Nicolas Ballas SSL 81 320 0 14 Apr 2022
On the Importance of Asymmetry for Siamese Representation Learning Tianlin Li Haoqi Fan Yuandong Tian Daisuke Kihara Xinlei Chen SSL 84 50 0 01 Apr 2022
How Does SimSiam Avoid Collapse Without Negative Samples? A Unified Understanding with Self-supervised Contrastive Learning Chaoning Zhang Kang Zhang Chenshuang Zhang T. Pham Chang D. Yoo In So Kweon SSL 52 73 0 30 Mar 2022
Contrasting the landscape of contrastive and non-contrastive learning Ashwini Pokle Jinjin Tian Yuchen Li Andrej Risteski SSL 73 28 0 29 Mar 2022
Exploring the Equivalence of Siamese Self-Supervised Learning via A Unified Gradient Framework Chenxin Tao Honghui Wang Xizhou Zhu Jiahua Dong S. Song Gao Huang Jifeng Dai SSL 52 59 0 09 Dec 2021
Understanding Dimensional Collapse in Contrastive Self-supervised Learning Li Jing Pascal Vincent Yann LeCun Yuandong Tian SSL 115 349 0 18 Oct 2021
Towards Demystifying Representation Learning with Non-contrastive Self-supervision Xiang Wang Xinlei Chen S. Du Yuandong Tian SSL 50 26 0 11 Oct 2021
VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning Adrien Bardes Jean Ponce Yann LeCun SSL DML 149 932 0 11 May 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 613 6,059 0 29 Apr 2021
Barlow Twins: Self-Supervised Learning via Redundancy Reduction Jure Zbontar Li Jing Ishan Misra Yann LeCun Stéphane Deny SSL 296 2,343 0 04 Mar 2021
Understanding self-supervised Learning Dynamics without Contrastive Pairs Yuandong Tian Xinlei Chen Surya Ganguli SSL 196 283 0 12 Feb 2021
Exploring Simple Siamese Representation Learning Xinlei Chen Kaiming He SSL 247 4,047 0 20 Nov 2020
Intriguing Properties of Contrastive Losses Ting Chen Calvin Luo Lala Li 60 174 0 05 Nov 2020
BYOL works even without batch statistics Pierre Harvey Richemond Jean-Bastien Grill Florent Altché Corentin Tallec Florian Strub ... Samuel L. Smith Soham De Razvan Pascanu Bilal Piot Michal Valko SSL 283 115 0 20 Oct 2020
Whitening for Self-Supervised Representation Learning Aleksandr Ermolov Aliaksandr Siarohin E. Sangineto N. Sebe SSL 86 313 0 13 Jul 2020
Unsupervised Learning of Visual Features by Contrasting Cluster Assignments Mathilde Caron Ishan Misra Julien Mairal Priya Goyal Piotr Bojanowski Armand Joulin OCL SSL 215 4,070 0 17 Jun 2020
Bootstrap your own latent: A new approach to self-supervised Learning Jean-Bastien Grill Florian Strub Florent Altché Corentin Tallec Pierre Harvey Richemond ... M. G. Azar Bilal Piot Koray Kavukcuoglu Rémi Munos Michal Valko SSL 347 6,792 0 13 Jun 2020
What Makes for Good Views for Contrastive Learning? Yonglong Tian Chen Sun Ben Poole Dilip Krishnan Cordelia Schmid Phillip Isola SSL 87 1,327 0 20 May 2020
Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere Tongzhou Wang Phillip Isola SSL 146 1,826 0 20 May 2020
A Simple Framework for Contrastive Learning of Visual Representations Ting-Li Chen Simon Kornblith Mohammad Norouzi Geoffrey E. Hinton SSL 339 18,721 0 13 Feb 2020
Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent Jaehoon Lee Lechao Xiao S. Schoenholz Yasaman Bahri Roman Novak Jascha Narain Sohl-Dickstein Jeffrey Pennington 196 1,099 0 18 Feb 2019
Representation Learning with Contrastive Predictive Coding Aaron van den Oord Yazhe Li Oriol Vinyals DRL SSL 300 10,282 0 10 Jul 2018
Neural Tangent Kernel: Convergence and Generalization in Neural Networks Arthur Jacot Franck Gabriel Clément Hongler 252 3,194 0 20 Jun 2018
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.1K 193,814 0 10 Dec 2015
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks Andrew M. Saxe James L. McClelland Surya Ganguli ODL 162 1,843 0 20 Dec 2013