v1v2v3v4v5v6 (latest)

Understanding Self-supervised Learning with Dual Deep Networks

1 October 2020

ArXiv (abs)PDF HTML Github (293★)

Papers citing "Understanding Self-supervised Learning with Dual Deep Networks"

42 / 42 papers shown

Title
SUMO: Subspace-Aware Moment-Orthogonalization for Accelerating Memory-Efficient LLM Training Yehonathan Refael Guy Smorodinsky Tom Tirer Ofir Lindenbaum 48 0 0 30 May 2025
AdaRankGrad: Adaptive Gradient-Rank and Moments for Memory-Efficient LLMs Training and Fine-Tuning Yehonathan Refael Jonathan Svirsky Boris Shustin Wasim Huleihel Ofir Lindenbaum 106 4 0 31 Dec 2024
GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection Jiawei Zhao Zhenyu Zhang Beidi Chen Zhangyang Wang A. Anandkumar Yuandong Tian 122 230 0 06 Mar 2024
Modulate Your Spectrum in Self-Supervised Learning Xi Weng Yu-Li Ni Tengwei Song Jie Luo Rao Muhammad Anwer Salman Khan Fahad Shahbaz Khan Lei Huang 86 5 0 26 May 2023
The SSL Interplay: Augmentations, Inductive Bias, and Generalization Vivien A. Cabannes B. Kiani Randall Balestriero Yann LeCun A. Bietti SSL 97 33 0 06 Feb 2023
Learning the Relation between Similarity Loss and Clustering Loss in Self-Supervised Learning Jidong Ge YuXiang Liu Jie Gui Lanting Fang Ming Lin James T. Kwok LiGuo Huang B. Luo SSL 105 5 0 08 Jan 2023
Boosting Automatic COVID-19 Detection Performance with Self-Supervised Learning and Batch Knowledge Ensembling Guang Li Ren Togo Takahiro Ogawa Miki Haseyama SSL 68 8 0 19 Dec 2022
Homomorphic Self-Supervised Learning Thomas Anderson Keller Xavier Suau Luca Zappella SSL 68 2 0 15 Nov 2022
Augmentation Invariant Manifold Learning Shulei Wang 244 1 0 01 Nov 2022
Neural Eigenfunctions Are Structured Representation Learners Zhijie Deng Jiaxin Shi Hao Zhang Peng Cui Cewu Lu Jun Zhu 111 14 0 23 Oct 2022
Correlation between Alignment-Uniformity and Performance of Dense Contrastive Representations Jong Hak Moon Wonjae Kim Edward Choi 115 3 0 17 Oct 2022
Improving Self-Supervised Learning by Characterizing Idealized Representations Yann Dubois Tatsunori Hashimoto Stefano Ermon Percy Liang SSL 170 43 0 13 Sep 2022
An Empirical Study on Disentanglement of Negative-free Contrastive Learning Jinkun Cao Ruiqian Nai Qing Yang Jialei Huang Yang Gao CoGe DRL 68 10 0 09 Jun 2022
TriBYOL: Triplet BYOL for Self-Supervised Representation Learning Guang Li Ren Togo Takahiro Ogawa Miki Haseyama SSL 124 18 0 07 Jun 2022
Beyond Just Vision: A Review on Self-Supervised Representation Learning on Multimodal and Temporal Data Shohreh Deldari Hao Xue Aaqib Saeed Jiayuan He Daniel V. Smith Flora D. Salim AI4TS 77 37 0 06 Jun 2022
Self-Labeling Refinement for Robust Representation Learning with Bootstrap Your Own Latent Siddhant Garg Dhruval Jain SSL 138 0 0 09 Apr 2022
On the Importance of Asymmetry for Siamese Representation Learning Tianlin Li Haoqi Fan Yuandong Tian Daisuke Kihara Xinlei Chen SSL 133 52 0 01 Apr 2022
Contrasting the landscape of contrastive and non-contrastive learning Ashwini Pokle Jinjin Tian Yuchen Li Andrej Risteski SSL 96 28 0 29 Mar 2022
Investigating Power laws in Deep Representation Learning Arna Ghosh Arnab Kumar Mondal Kumar Krishna Agrawal Blake A. Richards SSL OOD 52 19 0 11 Feb 2022
Understanding The Robustness of Self-supervised Learning Through Topic Modeling Zeping Luo Shiyou Wu C. Weng Mo Zhou Rong Ge OOD SSL 82 3 0 02 Feb 2022
Understanding Deep Contrastive Learning via Coordinate-wise Optimization Yuandong Tian 178 35 0 29 Jan 2022
DR3: Value-Based Deep Reinforcement Learning Requires Explicit Regularization Aviral Kumar Rishabh Agarwal Tengyu Ma Aaron Courville George Tucker Sergey Levine OffRL 97 69 0 09 Dec 2021
Understanding the Role of Self-Supervised Learning in Out-of-Distribution Detection Task Jiuhai Chen Chen Zhu Bin Dai OODD 115 3 0 26 Oct 2021
Understanding Dimensional Collapse in Contrastive Self-supervised Learning Li Jing Pascal Vincent Yann LeCun Yuandong Tian SSL 138 362 0 18 Oct 2021
Inverse Problems Leveraging Pre-trained Contrastive Representations Sriram Ravula Georgios Smyrnis Matt Jordan A. Dimakis SSL 150 9 0 14 Oct 2021
Towards Demystifying Representation Learning with Non-contrastive Self-supervision Xiang Wang Xinlei Chen S. Du Yuandong Tian SSL 78 26 0 11 Oct 2021
From Canonical Correlation Analysis to Self-supervised Graph Neural Networks Hengrui Zhang Qitian Wu Junchi Yan David Wipf Philip S. Yu SSL 110 223 0 23 Jun 2021
Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss Jeff Z. HaoChen Colin Wei Adrien Gaidon Tengyu Ma SSL 109 323 0 08 Jun 2021
Toward Understanding the Feature Learning Process of Self-supervised Contrastive Learning Zixin Wen Yuanzhi Li SSL MLT 96 136 0 31 May 2021
Self-Supervised Learning for Gastritis Detection with Gastric X-ray Images Guang Li Ren Togo Takahiro Ogawa Miki Haseyama 81 12 0 07 Apr 2021
Self-supervised Graph Neural Networks without explicit negative sampling Zekarias T. Kefato Sarunas Girdzijauskas SSL 113 44 0 27 Mar 2021
Barlow Twins: Self-Supervised Learning via Redundancy Reduction Jure Zbontar Li Jing Ishan Misra Yann LeCun Stéphane Deny SSL 396 2,378 0 04 Mar 2021
Mine Your Own vieW: Self-Supervised Learning Through Across-Sample Prediction Mehdi Azabou M. G. Azar Ran Liu Chi-Heng Lin Erik C. Johnson ... Lindsey Kitchell Keith B. Hengen William R. Gray Roncal Michal Valko Eva L. Dyer AI4TS 105 57 0 19 Feb 2021
Improving Few-Shot Learning with Auxiliary Self-Supervised Pretext Tasks N. Simard Guillaume Lagrange SSL 39 5 0 24 Jan 2021
CASTing Your Model: Learning to Localize Improves Self-Supervised Representations Ramprasaath R. Selvaraju Karan Desai Justin Johnson Nikhil Naik SSL 103 80 0 08 Dec 2020
How Well Do Self-Supervised Models Transfer? Linus Ericsson Henry Gouk Timothy M. Hospedales SSL 132 278 0 26 Nov 2020
Intriguing Properties of Contrastive Losses Ting Chen Calvin Luo Lala Li 96 177 0 05 Nov 2020
BYOL works even without batch statistics Pierre Harvey Richemond Jean-Bastien Grill Florent Altché Corentin Tallec Florian Strub ... Samuel L. Smith Soham De Razvan Pascanu Bilal Piot Michal Valko SSL 321 115 0 20 Oct 2020
Hard Negative Mixing for Contrastive Learning Yannis Kalantidis Mert Bulent Sariyildiz Noé Pion Philippe Weinzaepfel Diane Larlus SSL 159 648 0 02 Oct 2020
Predicting What You Already Know Helps: Provable Self-Supervised Learning Jason D. Lee Qi Lei Nikunj Saunshi Jiacheng Zhuo SSL 141 190 0 03 Aug 2020
Whitening for Self-Supervised Representation Learning Aleksandr Ermolov Aliaksandr Siarohin E. Sangineto N. Sebe SSL 114 316 0 13 Jul 2020
Unsupervised Learning of Visual Features by Contrasting Cluster Assignments Mathilde Caron Ishan Misra Julien Mairal Priya Goyal Piotr Bojanowski Armand Joulin OCL SSL 355 4,115 0 17 Jun 2020