Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss

8 June 2021

Adrien Gaidon

Papers citing "Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss"

50 / 77 papers shown

Title
seq-JEPA: Autoregressive Predictive Learning of Invariant-Equivariant World Models Hafez Ghaemi Eilif Muller Shahab Bakhtiari 54 0 0 06 May 2025
Contextures: Representations from Contexts Runtian Zhai Kai Yang Che-Ping Tsai Burak Varici Zico Kolter Pradeep Ravikumar 122 0 0 02 May 2025
THESAURUS: Contrastive Graph Clustering by Swapping Fused Gromov-Wasserstein Couplings Bowen Deng Tong Wang Lele Fu Sheng Huang Chuan Chen Tao Zhang 87 0 0 17 Feb 2025
Structure-preserving contrastive learning for spatial time series Yiru Jiao S. Cranenburgh Simeon C. Calvert H. Lint AI4TS 92 0 0 10 Feb 2025
Enhancing Graph Self-Supervised Learning with Graph Interplay Xinjian Zhao Wei Pang Xiangru Jian Yaoyao Xu Chaolong Ying Tianshu Yu 56 0 0 17 Jan 2025
Infinite Width Limits of Self Supervised Neural Networks Maximilian Fleissner Gautham Govind Anil D. Ghoshdastidar SSL 151 0 0 17 Nov 2024
Past, Present, and Future of Sensor-Based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task H. Haresamudram Chi Ian Tang Sungho Suh P. Lukowicz Thomas Ploetz 76 2 0 11 Nov 2024
Investigating the Impact of Model Complexity in Large Language Models Jing Luo Huiyuan Wang Weiran Huang 39 0 0 01 Oct 2024
On the Role of Discrete Tokenization in Visual Representation Learning Tianqi Du Yifei Wang Yisen Wang 52 7 0 12 Jul 2024
Look Ahead or Look Around? A Theoretical Comparison Between Autoregressive and Masked Pretraining Qi Zhang Tianqi Du Haotian Huang Yifei Wang Yisen Wang 42 3 0 01 Jul 2024
InfoNCE: Identifying the Gap Between Theory and Practice E. Rusak Patrik Reizinger Attila Juhos Oliver Bringmann Roland S. Zimmermann Wieland Brendel 35 5 0 28 Jun 2024
Theoretical Analysis of Weak-to-Strong Generalization Hunter Lang David Sontag Aravindan Vijayaraghavan 25 20 0 25 May 2024
Better Representations via Adversarial Training in Pre-Training: A Theoretical Perspective Yue Xing Xiaofeng Lin Qifan Song Yi Tian Xu Belinda Zeng Guang Cheng SSL 26 0 0 26 Jan 2024
Preserving Silent Features for Domain Generalization Chujie Zhao Tianren Zhang Feng Chen 23 0 0 06 Jan 2024
LiDAR: Sensing Linear Probing Performance in Joint Embedding SSL Architectures Vimal Thilak Chen Huang Omid Saremi Laurent Dinh Hanlin Goh Preetum Nakkiran Josh Susskind Etai Littwin 23 7 0 07 Dec 2023
Towards a Unified Framework of Contrastive Learning for Disentangled Representations Stefan Matthes Zhiwei Han Hao Shen 34 4 0 08 Nov 2023
Graph Ranking Contrastive Learning: A Extremely Simple yet Efficient Method Yulan Hu Ouyang Sheng Jingyu Liu Ge Chen Zhirui Yang Junchen Wan Fuzheng Zhang Zhongyuan Wang Yong Liu 31 0 0 23 Oct 2023
Towards the Sparseness of Projection Head in Self-Supervised Learning Zeen Song Xingzhe Su Jingyao Wang Wenwen Qiang Changwen Zheng Gang Hua 36 3 0 18 Jul 2023
Understanding Contrastive Learning Through the Lens of Margins Daniel Rho Taesoo Kim Sooill Park Jaehyun Park Jaehan Park SSL 32 2 0 20 Jun 2023
Unraveling Projection Heads in Contrastive Learning: Insights from Expansion and Shrinkage Yu Gui Cong Ma Yiqiao Zhong 22 6 0 06 Jun 2023
Quantifying Representation Reliability in Self-Supervised Learning Models Young-Jin Park Hao Wang Shervin Ardeshir Navid Azizan SSL UQCV 34 3 0 31 May 2023
Spectal Harmonics: Bridging Spectral Embedding and Matrix Completion in Self-Supervised Learning Marina Munkhoeva Ivan V. Oseledets SSL 34 2 0 31 May 2023
Incomplete Multi-view Clustering via Diffusion Completion Sifan Fang DiffM 16 4 0 19 May 2023
How does Contrastive Learning Organize Images? Yunzhe Zhang Yao Lu Qi Xuan SSL 32 0 0 17 May 2023
Denoising Cosine Similarity: A Theory-Driven Approach for Efficient Representation Learning Takumi Nakagawa Y. Sanada Hiroki Waida Yuhui Zhang Yuichiro Wada K. Takanashi Tomonori Yamada Takafumi Kanamori DiffM 19 5 0 19 Apr 2023
On the Stepwise Nature of Self-Supervised Learning James B. Simon Maksis Knutins Liu Ziyin Daniel Geisz Abraham J. Fetterman Joshua Albrecht SSL 37 30 0 27 Mar 2023
Active Self-Supervised Learning: A Few Low-Cost Relationships Are All You Need Vivien A. Cabannes Léon Bottou Yann LeCun Randall Balestriero 48 13 0 27 Mar 2023
Contrastive Learning Is Spectral Clustering On Similarity Graph Zhi-Hao Tan Yifan Zhang Jingqin Yang Yang Yuan SSL 54 18 0 27 Mar 2023
Multi-view Feature Extraction based on Triple Contrastive Heads Hongjie Zhang 11 0 0 22 Mar 2023
Preventing Dimensional Collapse of Incomplete Multi-View Clustering via Direct Contrastive Learning Kaiwu Zhang Shiqiang Du Bao-Yu Liu Shengxia Gao 22 0 0 22 Mar 2023
On the Provable Advantage of Unsupervised Pretraining Jiawei Ge Shange Tang Jianqing Fan Chi Jin SSL 33 16 0 02 Mar 2023
Unsupervised Learning on a DIET: Datum IndEx as Target Free of Self-Supervision, Reconstruction, Projector Head Randall Balestriero 38 3 0 20 Feb 2023
Self-supervised learning of Split Invariant Equivariant representations Q. Garrido Laurent Najman Yann LeCun SSL 32 32 0 14 Feb 2023
Multi-view Feature Extraction based on Dual Contrastive Head Hongjie Zhang 18 1 0 08 Feb 2023
The SSL Interplay: Augmentations, Inductive Bias, and Generalization Vivien A. Cabannes B. Kiani Randall Balestriero Yann LeCun A. Bietti SSL 19 31 0 06 Feb 2023
Revisiting Discriminative vs. Generative Classifiers: Theory and Implications Chenyu Zheng Guoqiang Wu Fan Bao Yue Cao Chongxuan Li Jun Zhu BDL 30 30 0 05 Feb 2023
Deciphering the Projection Head: Representation Evaluation Self-supervised Learning Jiajun Ma Tianyang Hu Wei Cao 17 8 0 28 Jan 2023
GEDI: GEnerative and DIscriminative Training for Self-Supervised Learning Emanuele Sansone Robin Manhaeve SSL 20 9 0 27 Dec 2022
On minimal variations for unsupervised representation learning Vivien A. Cabannes A. Bietti Randall Balestriero SSL DRL 30 8 0 07 Nov 2022
Contrastive Value Learning: Implicit Models for Simple Offline RL Bogdan Mazoure Benjamin Eysenbach Ofir Nachum Jonathan Tompson SSL OffRL 38 7 0 03 Nov 2022
Augmentation Invariant Manifold Learning Shulei Wang 36 1 0 01 Nov 2022
Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models Hong Liu Sang Michael Xie Zhiyuan Li Tengyu Ma AI4CE 40 49 0 25 Oct 2022
The Curious Case of Benign Memorization Sotiris Anagnostidis Gregor Bachmann Lorenzo Noci Thomas Hofmann AAML 49 8 0 25 Oct 2022
Label Propagation with Weak Supervision Rattana Pukdee Dylan Sam Maria-Florina Balcan Pradeep Ravikumar 40 9 0 07 Oct 2022
RankMe: Assessing the downstream performance of pretrained self-supervised representations by their rank Q. Garrido Randall Balestriero Laurent Najman Yann LeCun SSL 53 73 0 05 Oct 2022
What shapes the loss landscape of self-supervised learning? Liu Ziyin Ekdeep Singh Lubana Masakuni Ueda Hidenori Tanaka 50 20 0 02 Oct 2022
Minimalistic Unsupervised Learning with the Sparse Manifold Transform Yubei Chen Zeyu Yun Y. Ma Bruno A. Olshausen Yann LeCun 52 8 0 30 Sep 2022
Variance Covariance Regularization Enforces Pairwise Independence in Self-Supervised Representations Grégoire Mialon Randall Balestriero Yann LeCun 32 9 0 29 Sep 2022
Joint Embedding Self-Supervised Learning in the Kernel Regime B. Kiani Randall Balestriero Yubei Chen S. Lloyd Yann LeCun SSL 49 13 0 29 Sep 2022
The Geometry of Self-supervised Learning Models and its Impact on Transfer Learning Romain Cosentino Sarath Shekkizhar Mahdi Soltanolkotabi A. Avestimehr Antonio Ortega SSL 54 7 0 18 Sep 2022