Understanding Contrastive Learning Requires Incorporating Inductive Biases

28 February 2022

Papers citing "Understanding Contrastive Learning Requires Incorporating Inductive Biases"

50 / 71 papers shown

Title
A Unified and Scalable Membership Inference Method for Visual Self-supervised Encoder via Part-aware Capability Jie Zhu Jirong Zha Ding Li Leye Wang 131 1 0 15 May 2025
CIBR: Cross-modal Information Bottleneck Regularization for Robust CLIP Generalization Yingrui Ji Xi Xiao Gaofei Chen Hao Xu Chenrui Ma Lijing Zhu Aokun Liang Jiansheng Chen VLM 82 0 0 31 Mar 2025
Cross-Lingual IPA Contrastive Learning for Zero-Shot NER Jimin Sohn David R. Mortensen 83 0 0 10 Mar 2025
Projection Head is Secretly an Information Bottleneck Zhuo Ouyang Kaiwen Hu Qi Zhang Yifei Wang Yisen Wang 110 2 0 01 Mar 2025
Your contrastive learning problem is secretly a distribution alignment problem Zihao Chen Chi-Heng Lin Ran Liu Jingyun Xiao Eva L. Dyer 137 1 0 27 Feb 2025
Reasoning with Latent Thoughts: On the Power of Looped Transformers Nikunj Saunshi Nishanth Dikkala Zhiyuan Li Sanjiv Kumar Sashank J. Reddi OffRL LRM AI4CE 141 22 0 24 Feb 2025
Understanding the Role of Equivariance in Self-supervised Learning Yifei Wang Kaiwen Hu Sharut Gupta Ziyu Ye Yisen Wang Stefanie Jegelka SSL 97 2 0 10 Nov 2024
Understanding Self-Supervised Learning via Gaussian Mixture Models Parikshit Bansal Ali Kavis Sujay Sanghavi SSL 74 0 0 05 Nov 2024
On the Inductive Bias of Stacking Towards Improving Reasoning Nikunj Saunshi Stefani Karp Shankar Krishnan Sobhan Miryoosefi Sashank J. Reddi Sanjiv Kumar LRM AI4CE 86 7 0 27 Sep 2024
InfoNCE: Identifying the Gap Between Theory and Practice E. Rusak Patrik Reizinger Attila Juhos Oliver Bringmann Roland S. Zimmermann Wieland Brendel 130 11 0 28 Jun 2024
The Hidden Pitfalls of the Cosine Similarity Loss Andrew Draganov Sharvaree P. Vadgama Erik J. Bekkers SSL 93 2 0 24 Jun 2024
Causal Contrastive Learning for Counterfactual Regression Over Time Mouad El Bouchattaoui Myriam Tami Benoit Lepetit P. Cournède CML AI4TS 83 3 0 01 Jun 2024
A Generalization Theory of Cross-Modality Distillation with Contrastive Learning Hangyu Lin Chen Liu Chengming Xu Zhengqi Gao Yanwei Fu Yuan Yao VLM 112 1 0 06 May 2024
Position: Understanding LLMs Requires More Than Statistical Generalization Patrik Reizinger Szilvia Ujváry Anna Mészáros A. Kerekes Wieland Brendel Ferenc Huszár 136 16 0 03 May 2024
Can Contrastive Learning Refine Embeddings Lihui Liu Jinha Kim Vidit Bansal SSL 71 1 0 11 Apr 2024
Uniform Memory Retrieval with Larger Capacity for Modern Hopfield Models Dennis Wu Jerry Yao-Chieh Hu Teng-Yun Hsiao Han Liu 106 34 0 04 Apr 2024
A Unified Membership Inference Method for Visual Self-supervised Encoder via Part-aware Capability Jie Zhu Jirong Zha Ding Li Leye Wang 92 8 0 03 Apr 2024
TransFusion: Contrastive Learning with Transformers Huanran Li Daniel Pimentel-Alarcón 59 0 0 27 Mar 2024
Do Generated Data Always Help Contrastive Learning? Yifei Wang Jizhe Zhang Yisen Wang DiffM 96 26 0 19 Mar 2024
Investigating the Benefits of Projection Head for Representation Learning Yihao Xue Eric Gan Jiayi Ni Siddharth Joshi Baharan Mirzasoleiman 91 14 0 18 Mar 2024
Unsupervised learning based object detection using Contrastive Learning Chandan Kumar Jansel Herrera-Gerena John Just Matthew J. Darr Ali Jannesari SSL 81 1 0 21 Feb 2024
Analysis of Using Sigmoid Loss for Contrastive Learning Chungpa Lee Joonhwan Chang Jy-yong Sohn 87 4 0 20 Feb 2024
Efficient Stagewise Pretraining via Progressive Subnetworks Abhishek Panigrahi Nikunj Saunshi Kaifeng Lyu Sobhan Miryoosefi Sashank J. Reddi Satyen Kale Sanjiv Kumar 65 6 0 08 Feb 2024
A Probabilistic Model behind Self-Supervised Learning Alice Bizeul Bernhard Schölkopf Carl Allen SSL 75 2 0 02 Feb 2024
Better Representations via Adversarial Training in Pre-Training: A Theoretical Perspective Yue Xing Xiaofeng Lin Qifan Song Yi Tian Xu Belinda Zeng Guang Cheng SSL 54 0 0 26 Jan 2024
Complementary Benefits of Contrastive Learning and Self-Training Under Distribution Shift Saurabh Garg Amrith Rajagopal Setlur Zachary Chase Lipton Sivaraman Balakrishnan Virginia Smith Aditi Raghunathan SSL 74 9 0 06 Dec 2023
Optimal Sample Complexity of Contrastive Learning Noga Alon Dmitrii Avdiukhin Dor Elboim Orr Fischer G. Yaroslavtsev SSL 71 7 0 01 Dec 2023
Architecture Matters: Uncovering Implicit Mechanisms in Graph Contrastive Learning Xiaojun Guo Yifei Wang Zeming Wei Yisen Wang 103 12 0 05 Nov 2023
Identifiable Contrastive Learning with Automatic Feature Importance Discovery Qi Zhang Yifei Wang Yisen Wang 70 13 0 29 Oct 2023
Self-supervision meets kernel graph neural models: From architecture to augmentations Jiawang Dan Ruofan Wu Yunpeng Liu Baokun Wang Changhua Meng ... Tianyi Zhang Ningtao Wang Xin Fu Qi Li Weiqiang Wang SSL 76 1 0 17 Oct 2023
Understanding the Robustness of Multi-modal Contrastive Learning to Distribution Shift Yihao Xue Siddharth Joshi Dang Nguyen Baharan Mirzasoleiman VLM 69 4 0 08 Oct 2023
Perfect Alignment May be Poisonous to Graph Contrastive Learning Jingyu Liu Huayi Tang Yong Liu 66 3 0 06 Oct 2023
Understanding Transferable Representation Learning and Zero-shot Transfer in CLIP Zixiang Chen Yihe Deng Yuanzhi Li Quanquan Gu VLM 104 12 0 02 Oct 2023
Information Flow in Self-Supervised Learning Zhiyuan Tan Jingqin Yang Weiran Huang Yang Yuan Yifan Zhang SSL 83 14 0 29 Sep 2023
Survey of Social Bias in Vision-Language Models Nayeon Lee Yejin Bang Holy Lovenia Samuel Cahyawijaya Wenliang Dai Pascale Fung VLM 126 19 0 24 Sep 2023
Invariant Learning via Probability of Sufficient and Necessary Causes Mengyue Yang Zhen Fang Yonggang Zhang Yali Du Furui Liu Jean-François Ton Jianhong Wang Jun Wang OODD 148 14 0 22 Sep 2023
Towards a Rigorous Analysis of Mutual Information in Contrastive Learning Kyungeun Lee Jaeill Kim Suhyun Kang Wonjong Rhee SSL 82 2 0 30 Aug 2023
Towards the Sparseness of Projection Head in Self-Supervised Learning Changwen Zheng Xingzhe Su Wenwen Qiang Jingyao Wang Changwen Zheng Gang Hua 96 3 0 18 Jul 2023
On the Generalization of Multi-modal Contrastive Learning Qi Zhang Yifei Wang Yisen Wang 79 26 0 07 Jun 2023
Unraveling Projection Heads in Contrastive Learning: Insights from Expansion and Shrinkage Yu Gui Cong Ma Yiqiao Zhong 63 8 0 06 Jun 2023
Understanding Augmentation-based Self-Supervised Representation Learning via RKHS Approximation and Regression Runtian Zhai Bing Liu Andrej Risteski Zico Kolter Pradeep Ravikumar SSL 117 10 0 01 Jun 2023
Matrix Information Theory for Self-Supervised Learning Yifan Zhang Zhi-Hao Tan Jingqin Yang Weiran Huang Yang Yuan SSL 119 19 0 27 May 2023
Which Features are Learnt by Contrastive Learning? On the Role of Simplicity Bias in Class Collapse and Feature Suppression Yihao Xue S. Joshi Eric Gan Pin-Yu Chen Baharan Mirzasoleiman SSL 109 27 0 25 May 2023
How does Contrastive Learning Organize Images? Yunzhe Zhang Yao Lu Qi Xuan SSL 64 1 0 17 May 2023
No Free Lunch in Self Supervised Representation Learning Ihab Bendidi Adrien Bardes E. Cohen Alexis Lamiable Guillaume Bollot Auguste Genovesio OOD 88 11 0 23 Apr 2023
On the Stepwise Nature of Self-Supervised Learning James B. Simon Maksis Knutins Liu Ziyin Daniel Geisz Abraham J. Fetterman Joshua Albrecht SSL 90 35 0 27 Mar 2023
On the Provable Advantage of Unsupervised Pretraining Jiawei Ge Shange Tang Jianqing Fan Chi Jin SSL 78 17 0 02 Mar 2023
The Trade-off between Universality and Label Efficiency of Representations from Contrastive Learning Zhenmei Shi Jiefeng Chen Kunyang Li Jayaram Raghuram Xi Wu Yingyu Liang S. Jha SSL 79 20 0 28 Feb 2023
Generalization Analysis for Contrastive Representation Learning Yunwen Lei Tianbao Yang Yiming Ying Ding-Xuan Zhou 72 11 0 24 Feb 2023
InfoNCE Loss Provably Learns Cluster-Preserving Representations Advait Parulekar Liam Collins Karthikeyan Shanmugam Aryan Mokhtari Sanjay Shakkottai SSL 112 25 0 15 Feb 2023