Understanding self-supervised Learning Dynamics without Contrastive Pairs

12 February 2021

Papers citing "Understanding self-supervised Learning Dynamics without Contrastive Pairs"

31 / 31 papers shown

Title
PhiNets: Brain-inspired Non-contrastive Learning Based on Temporal Prediction Hypothesis Satoki Ishikawa Makoto Yamada Han Bao Yuki Takezawa 103 0 0 23 May 2024
Exploring Simple Siamese Representation Learning Xinlei Chen Kaiming He SSL 181 3,992 0 20 Nov 2020
Contrastive learning, multi-view redundancy, and linear models Christopher Tosh A. Krishnamurthy Daniel J. Hsu SSL 52 164 0 24 Aug 2020
Predicting What You Already Know Helps: Provable Self-Supervised Learning Jason D. Lee Qi Lei Nikunj Saunshi Jiacheng Zhuo SSL 41 187 0 03 Aug 2020
Unsupervised Learning of Visual Features by Contrasting Cluster Assignments Mathilde Caron Ishan Misra Julien Mairal Priya Goyal Piotr Bojanowski Armand Joulin OCL SSL 154 4,051 0 17 Jun 2020
Bootstrap your own latent: A new approach to self-supervised Learning Jean-Bastien Grill Florian Strub Florent Altché Corentin Tallec Pierre Harvey Richemond ... M. G. Azar Bilal Piot Koray Kavukcuoglu Rémi Munos Michal Valko SSL 268 6,718 0 13 Jun 2020
Improved Baselines with Momentum Contrastive Learning Xinlei Chen Haoqi Fan Ross B. Girshick Kaiming He SSL 421 3,397 0 09 Mar 2020
A Simple Framework for Contrastive Learning of Visual Representations Ting-Li Chen Simon Kornblith Mohammad Norouzi Geoffrey E. Hinton SSL 208 18,607 0 13 Feb 2020
Self-Supervised Learning of Pretext-Invariant Representations Ishan Misra Laurens van der Maaten SSL VLM 72 1,448 0 04 Dec 2019
Momentum Contrast for Unsupervised Visual Representation Learning Kaiming He Haoqi Fan Yuxin Wu Saining Xie Ross B. Girshick SSL 109 12,007 0 13 Nov 2019
Contrastive Multiview Coding Yonglong Tian Dilip Krishnan Phillip Isola SSL 129 2,385 0 13 Jun 2019
Learning Representations by Maximizing Mutual Information Across Views Philip Bachman R. Devon Hjelm William Buchwalter SSL 153 1,463 0 03 Jun 2019
A Theoretical Analysis of Contrastive Unsupervised Representation Learning Sanjeev Arora H. Khandeparkar M. Khodak Orestis Plevrakis Nikunj Saunshi SSL 86 768 0 25 Feb 2019
Width Provably Matters in Optimization for Deep Linear Neural Networks S. Du Wei Hu 53 94 0 24 Jan 2019
Gradient Descent Finds Global Minima of Deep Neural Networks S. Du Jason D. Lee Haochuan Li Liwei Wang Masayoshi Tomizuka ODL 119 1,133 0 09 Nov 2018
A mathematical theory of semantic development in deep neural networks Andrew M. Saxe James L. McClelland Surya Ganguli 51 264 0 23 Oct 2018
An analytic theory of generalization dynamics and transfer learning in deep linear networks Andrew Kyle Lampinen Surya Ganguli OOD 40 128 0 27 Sep 2018
Representation Learning with Contrastive Predictive Coding Aaron van den Oord Yazhe Li Oriol Vinyals DRL SSL 225 10,152 0 10 Jul 2018
Algorithmic Regularization in Learning Deep Homogeneous Models: Layers are Automatically Balanced S. Du Wei Hu Jason D. Lee MLT 114 239 0 04 Jun 2018
The Emergence of Spectral Universality in Deep Networks Jeffrey Pennington S. Schoenholz Surya Ganguli 44 171 0 27 Feb 2018
On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization Sanjeev Arora Nadav Cohen Elad Hazan 83 481 0 19 Feb 2018
Gradient descent with identity initialization efficiently learns positive definite linear transformations by deep residual networks Peter L. Bartlett D. Helmbold Philip M. Long 64 116 0 16 Feb 2018
Spurious Local Minima are Common in Two-Layer ReLU Neural Networks Itay Safran Ohad Shamir 119 263 0 24 Dec 2017
Deep linear neural networks with arbitrary loss: All local minima are global T. Laurent J. V. Brecht ODL 34 136 0 05 Dec 2017
Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice Jeffrey Pennington S. Schoenholz Surya Ganguli ODL 22 251 0 13 Nov 2017
Large Batch Training of Convolutional Networks Yang You Igor Gitman Boris Ginsburg ODL 106 844 0 13 Aug 2017
An Analytical Formula of Population Gradient for two-layered ReLU network and its Applications in Convergence and Critical Point Analysis Yuandong Tian MLT 114 216 0 02 Mar 2017
Globally Optimal Gradient Descent for a ConvNet with Gaussian Inputs Alon Brutzkus Amir Globerson MLT 97 313 0 26 Feb 2017
Deep Learning without Poor Local Minima Kenji Kawaguchi ODL 149 922 0 23 May 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 1.4K 192,638 0 10 Dec 2015
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks Andrew M. Saxe James L. McClelland Surya Ganguli ODL 111 1,830 0 20 Dec 2013