Understanding Contrastive Learning Requires Incorporating Inductive Biases

28 February 2022

Papers citing "Understanding Contrastive Learning Requires Incorporating Inductive Biases"

50 / 75 papers shown

Title
A Unified and Scalable Membership Inference Method for Visual Self-supervised Encoder via Part-aware Capability Jie Zhu Jirong Zha Ding Li Leye Wang 31 0 0 15 May 2025
CIBR: Cross-modal Information Bottleneck Regularization for Robust CLIP Generalization Yingrui Ji Xi Xiao Gaofei Chen Hao Xu Chenrui Ma Lijing Zhu Aokun Liang Jiansheng Chen VLM 53 0 0 31 Mar 2025
Cross-Lingual IPA Contrastive Learning for Zero-Shot NER Jimin Sohn David R. Mortensen 49 0 0 10 Mar 2025
Projection Head is Secretly an Information Bottleneck Zhuo Ouyang Kaiwen Hu Qi Zhang Yifei Wang Yisen Wang 48 0 0 01 Mar 2025
Your contrastive learning problem is secretly a distribution alignment problem Zihao Chen Chi-Heng Lin Ran Liu Jingyun Xiao Eva L. Dyer 75 1 0 27 Feb 2025
Reasoning with Latent Thoughts: On the Power of Looped Transformers Nikunj Saunshi Nishanth Dikkala Zhiyuan Li Sanjiv Kumar Sashank J. Reddi OffRL LRM AI4CE 58 10 0 24 Feb 2025
Understanding the Role of Equivariance in Self-supervised Learning Yifei Wang Kaiwen Hu Sharut Gupta Ziyu Ye Yisen Wang Stefanie Jegelka SSL 44 2 0 10 Nov 2024
Understanding Self-Supervised Learning via Gaussian Mixture Models Parikshit Bansal Ali Kavis Sujay Sanghavi SSL 37 0 0 05 Nov 2024
On the Inductive Bias of Stacking Towards Improving Reasoning Nikunj Saunshi Stefani Karp Shankar Krishnan Sobhan Miryoosefi Sashank J. Reddi Sanjiv Kumar LRM AI4CE 34 4 0 27 Sep 2024
InfoNCE: Identifying the Gap Between Theory and Practice E. Rusak Patrik Reizinger Attila Juhos Oliver Bringmann Roland S. Zimmermann Wieland Brendel 33 5 0 28 Jun 2024
The Hidden Pitfalls of the Cosine Similarity Loss Andrew Draganov Sharvaree P. Vadgama Erik J. Bekkers SSL 31 1 0 24 Jun 2024
Causal Contrastive Learning for Counterfactual Regression Over Time Mouad El Bouchattaoui Myriam Tami Benoit Lepetit P. Cournède CML AI4TS 38 2 0 01 Jun 2024
A Generalization Theory of Cross-Modality Distillation with Contrastive Learning Hangyu Lin Chen Liu Chengming Xu Zhengqi Gao Yanwei Fu Yuan Yao VLM 38 0 0 06 May 2024
Position: Understanding LLMs Requires More Than Statistical Generalization Patrik Reizinger Szilvia Ujváry Anna Mészáros A. Kerekes Wieland Brendel Ferenc Huszár 36 12 0 03 May 2024
Can Contrastive Learning Refine Embeddings Lihui Liu Jinha Kim Vidit Bansal SSL 34 1 0 11 Apr 2024
Uniform Memory Retrieval with Larger Capacity for Modern Hopfield Models Dennis Wu Jerry Yao-Chieh Hu Teng-Yun Hsiao Han Liu 40 28 0 04 Apr 2024
A Unified Membership Inference Method for Visual Self-supervised Encoder via Part-aware Capability Jie Zhu Jirong Zha Ding Li Leye Wang 37 6 0 03 Apr 2024
TransFusion: Contrastive Learning with Transformers Huanran Li Daniel Pimentel-Alarcón 36 0 0 27 Mar 2024
Do Generated Data Always Help Contrastive Learning? Yifei Wang Jizhe Zhang Yisen Wang DiffM 30 22 0 19 Mar 2024
Investigating the Benefits of Projection Head for Representation Learning Yihao Xue Eric Gan Jiayi Ni Siddharth Joshi Baharan Mirzasoleiman 41 10 0 18 Mar 2024
Unsupervised learning based object detection using Contrastive Learning Chandan Kumar Jansel Herrera-Gerena John Just Matthew J. Darr Ali Jannesari SSL 51 1 0 21 Feb 2024
Analysis of Using Sigmoid Loss for Contrastive Learning Chungpa Lee Joonhwan Chang Jy-yong Sohn 48 2 0 20 Feb 2024
Efficient Stagewise Pretraining via Progressive Subnetworks Abhishek Panigrahi Nikunj Saunshi Kaifeng Lyu Sobhan Miryoosefi Sashank J. Reddi Satyen Kale Sanjiv Kumar 38 5 0 08 Feb 2024
A Probabilistic Model behind Self-Supervised Learning Alice Bizeul Bernhard Schölkopf Carl Allen SSL 26 2 0 02 Feb 2024
Better Representations via Adversarial Training in Pre-Training: A Theoretical Perspective Yue Xing Xiaofeng Lin Qifan Song Yi Tian Xu Belinda Zeng Guang Cheng SSL 23 0 0 26 Jan 2024
Complementary Benefits of Contrastive Learning and Self-Training Under Distribution Shift Saurabh Garg Amrith Rajagopal Setlur Zachary Chase Lipton Sivaraman Balakrishnan Virginia Smith Aditi Raghunathan SSL 30 6 0 06 Dec 2023
Optimal Sample Complexity of Contrastive Learning Noga Alon Dmitrii Avdiukhin Dor Elboim Orr Fischer G. Yaroslavtsev SSL 24 5 0 01 Dec 2023
Architecture Matters: Uncovering Implicit Mechanisms in Graph Contrastive Learning Xiaojun Guo Yifei Wang Zeming Wei Yisen Wang 33 11 0 05 Nov 2023
Identifiable Contrastive Learning with Automatic Feature Importance Discovery Qi Zhang Yifei Wang Yisen Wang 31 12 0 29 Oct 2023
Self-supervision meets kernel graph neural models: From architecture to augmentations Jiawang Dan Ruofan Wu Yunpeng Liu Baokun Wang Changhua Meng ... Tianyi Zhang Ningtao Wang Xin Fu Qi Li Weiqiang Wang SSL 31 1 0 17 Oct 2023
Understanding the Robustness of Multi-modal Contrastive Learning to Distribution Shift Yihao Xue Siddharth Joshi Dang Nguyen Baharan Mirzasoleiman VLM 31 4 0 08 Oct 2023
Perfect Alignment May be Poisonous to Graph Contrastive Learning Jingyu Liu Huayi Tang Yong Liu 28 2 0 06 Oct 2023
Understanding Transferable Representation Learning and Zero-shot Transfer in CLIP Zixiang Chen Yihe Deng Yuanzhi Li Quanquan Gu VLM 23 10 0 02 Oct 2023
Information Flow in Self-Supervised Learning Zhiyuan Tan Jingqin Yang Weiran Huang Yang Yuan Yifan Zhang SSL 30 13 0 29 Sep 2023
Survey of Social Bias in Vision-Language Models Nayeon Lee Yejin Bang Holy Lovenia Samuel Cahyawijaya Wenliang Dai Pascale Fung VLM 44 16 0 24 Sep 2023
Invariant Learning via Probability of Sufficient and Necessary Causes Mengyue Yang Zhen Fang Yonggang Zhang Yali Du Furui Liu Jean-François Ton Jianhong Wang Jun Wang OODD 40 13 0 22 Sep 2023
Towards a Rigorous Analysis of Mutual Information in Contrastive Learning Kyungeun Lee Jaeill Kim Suhyun Kang Wonjong Rhee SSL 33 2 0 30 Aug 2023
Towards the Sparseness of Projection Head in Self-Supervised Learning Changwen Zheng Xingzhe Su Wenwen Qiang Jingyao Wang Changwen Zheng Gang Hua 36 3 0 18 Jul 2023
On the Generalization of Multi-modal Contrastive Learning Qi Zhang Yifei Wang Yisen Wang 22 24 0 07 Jun 2023
Unraveling Projection Heads in Contrastive Learning: Insights from Expansion and Shrinkage Yu Gui Cong Ma Yiqiao Zhong 22 6 0 06 Jun 2023
Understanding Augmentation-based Self-Supervised Representation Learning via RKHS Approximation and Regression Runtian Zhai Bing Liu Andrej Risteski Zico Kolter Pradeep Ravikumar SSL 28 9 0 01 Jun 2023
Matrix Information Theory for Self-Supervised Learning Yifan Zhang Zhi-Hao Tan Jingqin Yang Weiran Huang Yang Yuan SSL 45 16 0 27 May 2023
Which Features are Learnt by Contrastive Learning? On the Role of Simplicity Bias in Class Collapse and Feature Suppression Yihao Xue S. Joshi Eric Gan Pin-Yu Chen Baharan Mirzasoleiman SSL 30 23 0 25 May 2023
How does Contrastive Learning Organize Images? Yunzhe Zhang Yao Lu Qi Xuan SSL 29 0 0 17 May 2023
No Free Lunch in Self Supervised Representation Learning Ihab Bendidi Adrien Bardes E. Cohen Alexis Lamiable Guillaume Bollot Auguste Genovesio OOD 49 11 0 23 Apr 2023
On the Stepwise Nature of Self-Supervised Learning James B. Simon Maksis Knutins Liu Ziyin Daniel Geisz Abraham J. Fetterman Joshua Albrecht SSL 34 30 0 27 Mar 2023
On the Provable Advantage of Unsupervised Pretraining Jiawei Ge Shange Tang Jianqing Fan Chi Jin SSL 33 16 0 02 Mar 2023
The Trade-off between Universality and Label Efficiency of Representations from Contrastive Learning Zhenmei Shi Jiefeng Chen Kunyang Li Jayaram Raghuram Xi Wu Yingyu Liang S. Jha SSL 22 17 0 28 Feb 2023
Generalization Analysis for Contrastive Representation Learning Yunwen Lei Tianbao Yang Yiming Ying Ding-Xuan Zhou 28 8 0 24 Feb 2023
InfoNCE Loss Provably Learns Cluster-Preserving Representations Advait Parulekar Liam Collins Karthikeyan Shanmugam Aryan Mokhtari Sanjay Shakkottai SSL 34 20 0 15 Feb 2023