Papers
Communities
Events
Blog
Pricing
Search
Open menu
Home
Papers
2010.00578
Cited By
v1
v2
v3
v4
v5
v6 (latest)
Understanding Self-supervised Learning with Dual Deep Networks
1 October 2020
Yuandong Tian
Lantao Yu
Xinlei Chen
Surya Ganguli
SSL
Re-assign community
ArXiv (abs)
PDF
HTML
Github (293★)
Papers citing
"Understanding Self-supervised Learning with Dual Deep Networks"
42 / 42 papers shown
Title
SUMO: Subspace-Aware Moment-Orthogonalization for Accelerating Memory-Efficient LLM Training
Yehonathan Refael
Guy Smorodinsky
Tom Tirer
Ofir Lindenbaum
48
0
0
30 May 2025
AdaRankGrad: Adaptive Gradient-Rank and Moments for Memory-Efficient LLMs Training and Fine-Tuning
Yehonathan Refael
Jonathan Svirsky
Boris Shustin
Wasim Huleihel
Ofir Lindenbaum
106
4
0
31 Dec 2024
GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection
Jiawei Zhao
Zhenyu Zhang
Beidi Chen
Zhangyang Wang
A. Anandkumar
Yuandong Tian
122
230
0
06 Mar 2024
Modulate Your Spectrum in Self-Supervised Learning
Xi Weng
Yu-Li Ni
Tengwei Song
Jie Luo
Rao Muhammad Anwer
Salman Khan
Fahad Shahbaz Khan
Lei Huang
86
5
0
26 May 2023
The SSL Interplay: Augmentations, Inductive Bias, and Generalization
Vivien A. Cabannes
B. Kiani
Randall Balestriero
Yann LeCun
A. Bietti
SSL
97
33
0
06 Feb 2023
Learning the Relation between Similarity Loss and Clustering Loss in Self-Supervised Learning
Jidong Ge
YuXiang Liu
Jie Gui
Lanting Fang
Ming Lin
James T. Kwok
LiGuo Huang
B. Luo
SSL
105
5
0
08 Jan 2023
Boosting Automatic COVID-19 Detection Performance with Self-Supervised Learning and Batch Knowledge Ensembling
Guang Li
Ren Togo
Takahiro Ogawa
Miki Haseyama
SSL
68
8
0
19 Dec 2022
Homomorphic Self-Supervised Learning
Thomas Anderson Keller
Xavier Suau
Luca Zappella
SSL
68
2
0
15 Nov 2022
Augmentation Invariant Manifold Learning
Shulei Wang
244
1
0
01 Nov 2022
Neural Eigenfunctions Are Structured Representation Learners
Zhijie Deng
Jiaxin Shi
Hao Zhang
Peng Cui
Cewu Lu
Jun Zhu
115
14
0
23 Oct 2022
Correlation between Alignment-Uniformity and Performance of Dense Contrastive Representations
Jong Hak Moon
Wonjae Kim
Edward Choi
115
3
0
17 Oct 2022
Improving Self-Supervised Learning by Characterizing Idealized Representations
Yann Dubois
Tatsunori Hashimoto
Stefano Ermon
Percy Liang
SSL
170
43
0
13 Sep 2022
An Empirical Study on Disentanglement of Negative-free Contrastive Learning
Jinkun Cao
Ruiqian Nai
Qing Yang
Jialei Huang
Yang Gao
CoGe
DRL
68
10
0
09 Jun 2022
TriBYOL: Triplet BYOL for Self-Supervised Representation Learning
Guang Li
Ren Togo
Takahiro Ogawa
Miki Haseyama
SSL
124
18
0
07 Jun 2022
Beyond Just Vision: A Review on Self-Supervised Representation Learning on Multimodal and Temporal Data
Shohreh Deldari
Hao Xue
Aaqib Saeed
Jiayuan He
Daniel V. Smith
Flora D. Salim
AI4TS
77
37
0
06 Jun 2022
Self-Labeling Refinement for Robust Representation Learning with Bootstrap Your Own Latent
Siddhant Garg
Dhruval Jain
SSL
140
0
0
09 Apr 2022
On the Importance of Asymmetry for Siamese Representation Learning
Tianlin Li
Haoqi Fan
Yuandong Tian
Daisuke Kihara
Xinlei Chen
SSL
133
52
0
01 Apr 2022
Contrasting the landscape of contrastive and non-contrastive learning
Ashwini Pokle
Jinjin Tian
Yuchen Li
Andrej Risteski
SSL
96
28
0
29 Mar 2022
Investigating Power laws in Deep Representation Learning
Arna Ghosh
Arnab Kumar Mondal
Kumar Krishna Agrawal
Blake A. Richards
SSL
OOD
52
19
0
11 Feb 2022
Understanding The Robustness of Self-supervised Learning Through Topic Modeling
Zeping Luo
Shiyou Wu
C. Weng
Mo Zhou
Rong Ge
OOD
SSL
82
3
0
02 Feb 2022
Understanding Deep Contrastive Learning via Coordinate-wise Optimization
Yuandong Tian
178
35
0
29 Jan 2022
DR3: Value-Based Deep Reinforcement Learning Requires Explicit Regularization
Aviral Kumar
Rishabh Agarwal
Tengyu Ma
Aaron Courville
George Tucker
Sergey Levine
OffRL
97
69
0
09 Dec 2021
Understanding the Role of Self-Supervised Learning in Out-of-Distribution Detection Task
Jiuhai Chen
Chen Zhu
Bin Dai
OODD
115
3
0
26 Oct 2021
Understanding Dimensional Collapse in Contrastive Self-supervised Learning
Li Jing
Pascal Vincent
Yann LeCun
Yuandong Tian
SSL
138
362
0
18 Oct 2021
Inverse Problems Leveraging Pre-trained Contrastive Representations
Sriram Ravula
Georgios Smyrnis
Matt Jordan
A. Dimakis
SSL
150
9
0
14 Oct 2021
Towards Demystifying Representation Learning with Non-contrastive Self-supervision
Xiang Wang
Xinlei Chen
S. Du
Yuandong Tian
SSL
78
26
0
11 Oct 2021
From Canonical Correlation Analysis to Self-supervised Graph Neural Networks
Hengrui Zhang
Qitian Wu
Junchi Yan
David Wipf
Philip S. Yu
SSL
110
223
0
23 Jun 2021
Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss
Jeff Z. HaoChen
Colin Wei
Adrien Gaidon
Tengyu Ma
SSL
111
323
0
08 Jun 2021
Toward Understanding the Feature Learning Process of Self-supervised Contrastive Learning
Zixin Wen
Yuanzhi Li
SSL
MLT
96
136
0
31 May 2021
Self-Supervised Learning for Gastritis Detection with Gastric X-ray Images
Guang Li
Ren Togo
Takahiro Ogawa
Miki Haseyama
81
12
0
07 Apr 2021
Self-supervised Graph Neural Networks without explicit negative sampling
Zekarias T. Kefato
Sarunas Girdzijauskas
SSL
113
44
0
27 Mar 2021
Barlow Twins: Self-Supervised Learning via Redundancy Reduction
Jure Zbontar
Li Jing
Ishan Misra
Yann LeCun
Stéphane Deny
SSL
396
2,379
0
04 Mar 2021
Mine Your Own vieW: Self-Supervised Learning Through Across-Sample Prediction
Mehdi Azabou
M. G. Azar
Ran Liu
Chi-Heng Lin
Erik C. Johnson
...
Lindsey Kitchell
Keith B. Hengen
William R. Gray Roncal
Michal Valko
Eva L. Dyer
AI4TS
105
57
0
19 Feb 2021
Improving Few-Shot Learning with Auxiliary Self-Supervised Pretext Tasks
N. Simard
Guillaume Lagrange
SSL
39
5
0
24 Jan 2021
CASTing Your Model: Learning to Localize Improves Self-Supervised Representations
Ramprasaath R. Selvaraju
Karan Desai
Justin Johnson
Nikhil Naik
SSL
103
80
0
08 Dec 2020
How Well Do Self-Supervised Models Transfer?
Linus Ericsson
Henry Gouk
Timothy M. Hospedales
SSL
132
278
0
26 Nov 2020
Intriguing Properties of Contrastive Losses
Ting Chen
Calvin Luo
Lala Li
100
177
0
05 Nov 2020
BYOL works even without batch statistics
Pierre Harvey Richemond
Jean-Bastien Grill
Florent Altché
Corentin Tallec
Florian Strub
...
Samuel L. Smith
Soham De
Razvan Pascanu
Bilal Piot
Michal Valko
SSL
321
115
0
20 Oct 2020
Hard Negative Mixing for Contrastive Learning
Yannis Kalantidis
Mert Bulent Sariyildiz
Noé Pion
Philippe Weinzaepfel
Diane Larlus
SSL
159
648
0
02 Oct 2020
Predicting What You Already Know Helps: Provable Self-Supervised Learning
Jason D. Lee
Qi Lei
Nikunj Saunshi
Jiacheng Zhuo
SSL
141
190
0
03 Aug 2020
Whitening for Self-Supervised Representation Learning
Aleksandr Ermolov
Aliaksandr Siarohin
E. Sangineto
N. Sebe
SSL
114
316
0
13 Jul 2020
Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
Mathilde Caron
Ishan Misra
Julien Mairal
Priya Goyal
Piotr Bojanowski
Armand Joulin
OCL
SSL
355
4,115
0
17 Jun 2020
1