Can contrastive learning avoid shortcut solutions?

21 June 2021

Kayhan Batmanghelich

Papers citing "Can contrastive learning avoid shortcut solutions?"

50 / 58 papers shown

Title
Your contrastive learning problem is secretly a distribution alignment problem Zihao Chen Chi-Heng Lin Ran Liu Jingyun Xiao Eva L. Dyer 88 1 0 27 Feb 2025
Unleashing the Potential of Pre-Trained Diffusion Models for Generalizable Person Re-Identification Jiachen Li Xiaojin Gong DiffM 113 0 0 10 Feb 2025
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training Sanghwan Kim Rui Xiao Mariana-Iuliana Georgescu Stephan Alaniz Zeynep Akata VLM 168 2 0 02 Dec 2024
Banyan: Improved Representation Learning with Explicit Structure Mattia Opper N. Siddharth 76 1 0 25 Jul 2024
InfoNCE: Identifying the Gap Between Theory and Practice E. Rusak Patrik Reizinger Attila Juhos Oliver Bringmann Roland S. Zimmermann Wieland Brendel 70 7 0 28 Jun 2024
Unsupervised Multi-modal Feature Alignment for Time Series Representation Learning Cheng Liang Donghua Yang Zhiyu Liang Hongzhi Wang Zheng Liang Xiyang Zhang Jianfeng Huang AI4TS 351 1 0 09 Dec 2023
Augmenting Chest X-ray Datasets with Non-Expert Annotations Veronika Cheplygina Cathrine Damgaard Dovile Juodelyte Veronika Cheplygina Amelia Jiménez-Sánchez 80 3 0 05 Sep 2023
VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning Adrien Bardes Jean Ponce Yann LeCun SSL DML 92 917 0 11 May 2021
Contrastive Learning with Stronger Augmentations Tianlin Li Guo-Jun Qi CLL 79 224 0 15 Apr 2021
The Low-Rank Simplicity Bias in Deep Networks Minyoung Huh H. Mobahi Richard Y. Zhang Brian Cheung Pulkit Agrawal Phillip Isola 47 113 0 18 Mar 2021
Barlow Twins: Self-Supervised Learning via Redundancy Reduction Jure Zbontar Li Jing Ishan Misra Yann LeCun Stéphane Deny SSL 191 2,321 0 04 Mar 2021
Contrastive Learning Inverts the Data Generating Process Roland S. Zimmermann Yash Sharma Steffen Schneider Matthias Bethge Wieland Brendel SSL 256 215 0 17 Feb 2021
Understanding the Behaviour of Contrastive Loss Feng Wang Huaping Liu SSL 65 679 0 15 Dec 2020
Context Matters: Graph-based Self-supervised Representation Learning for Medical Images Li Sun Ke Yu Kayhan Batmanghelich SSL 47 23 0 11 Dec 2020
Exploring Simple Siamese Representation Learning Xinlei Chen Kaiming He SSL 154 3,992 0 20 Nov 2020
AdCo: Adversarial Contrast for Efficient Learning of Unsupervised Representations from Self-Trained Negative Adversaries Q. Hu Tianlin Li Wei Hu Guo-Jun Qi SSL 28 153 0 17 Nov 2020
Intriguing Properties of Contrastive Losses Ting Chen Calvin Luo Lala Li 40 174 0 05 Nov 2020
Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depth Thao Nguyen M. Raghu Simon Kornblith OOD 26 271 0 29 Oct 2020
Robust Pre-Training by Adversarial Contrastive Learning Ziyu Jiang Tianlong Chen Ting-Li Chen Zhangyang Wang 62 230 0 26 Oct 2020
Contrastive Learning with Adversarial Examples Chih-Hui Ho Nuno Vasconcelos SSL 34 142 0 22 Oct 2020
Contrastive Learning with Hard Negative Samples Joshua Robinson Ching-Yao Chuang S. Sra Stefanie Jegelka SSL 102 768 0 09 Oct 2020
Unsupervised Representation Learning by InvariancePropagation Feng Wang Huaping Liu Di Guo F. Sun SSL 45 33 0 07 Oct 2020
Hard Negative Mixing for Contrastive Learning Yannis Kalantidis Mert Bulent Sariyildiz Noé Pion Philippe Weinzaepfel Diane Larlus SSL 81 634 0 02 Oct 2020
Predicting What You Already Know Helps: Provable Self-Supervised Learning Jason D. Lee Qi Lei Nikunj Saunshi Jiacheng Zhuo SSL 41 187 0 03 Aug 2020
Debiased Contrastive Learning Ching-Yao Chuang Joshua Robinson Yen-Chen Lin Antonio Torralba Stefanie Jegelka SSL 35 558 0 01 Jul 2020
What shapes feature representations? Exploring datasets, architectures, and training Katherine L. Hermann Andrew Kyle Lampinen OOD 37 156 0 22 Jun 2020
Unsupervised Learning of Visual Features by Contrasting Cluster Assignments Mathilde Caron Ishan Misra Julien Mairal Priya Goyal Piotr Bojanowski Armand Joulin OCL SSL 106 4,032 0 17 Jun 2020
Bootstrap your own latent: A new approach to self-supervised Learning Jean-Bastien Grill Florian Strub Florent Altché Corentin Tallec Pierre Harvey Richemond ... M. G. Azar Bilal Piot Koray Kavukcuoglu Rémi Munos Michal Valko SSL 247 6,718 0 13 Jun 2020
Adversarial Self-Supervised Contrastive Learning Minseon Kim Jihoon Tack Sung Ju Hwang SSL 47 249 0 13 Jun 2020
What makes instance discrimination good for transfer learning? Nanxuan Zhao Zhirong Wu Rynson W. H. Lau Stephen Lin SSL 46 168 0 11 Jun 2020
Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere Tongzhou Wang Phillip Isola SSL 74 1,808 0 20 May 2020
What Makes for Good Views for Contrastive Learning? Yonglong Tian Chen Sun Ben Poole Dilip Krishnan Cordelia Schmid Phillip Isola SSL 63 1,313 0 20 May 2020
Shortcut Learning in Deep Neural Networks Robert Geirhos J. Jacobsen Claudio Michaelis R. Zemel Wieland Brendel Matthias Bethge Felix Wichmann 131 2,023 0 16 Apr 2020
Improved Baselines with Momentum Contrastive Learning Xinlei Chen Haoqi Fan Ross B. Girshick Kaiming He SSL 408 3,397 0 09 Mar 2020
Automatic Shortcut Removal for Self-Supervised Representation Learning Matthias Minderer Olivier Bachem N. Houlsby Michael Tschannen SSL 39 73 0 20 Feb 2020
Strength from Weakness: Fast Learning Using Weak Supervision Joshua Robinson Stefanie Jegelka S. Sra 52 32 0 19 Feb 2020
A Simple Framework for Contrastive Learning of Visual Representations Ting-Li Chen Simon Kornblith Mohammad Norouzi Geoffrey E. Hinton SSL 158 18,523 0 13 Feb 2020
Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Loss Lénaïc Chizat Francis R. Bach MLT 56 332 0 11 Feb 2020
PyTorch: An Imperative Style, High-Performance Deep Learning Library Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury ... Sasank Chilamkurthy Benoit Steiner Lu Fang Junjie Bai Soumith Chintala ODL 134 42,038 0 03 Dec 2019
Momentum Contrast for Unsupervised Visual Representation Learning Kaiming He Haoqi Fan Yuxin Wu Saining Xie Ross B. Girshick SSL 83 11,959 0 13 Nov 2019
Gradient Descent Maximizes the Margin of Homogeneous Neural Networks Kaifeng Lyu Jian Li 68 332 0 13 Jun 2019
Contrastive Multiview Coding Yonglong Tian Dilip Krishnan Phillip Isola SSL 123 2,385 0 13 Jun 2019
Adversarial Examples Are Not Bugs, They Are Features Andrew Ilyas Shibani Santurkar Dimitris Tsipras Logan Engstrom Brandon Tran Aleksander Madry SILM 72 1,825 0 06 May 2019
Do ImageNet Classifiers Generalize to ImageNet? Benjamin Recht Rebecca Roelofs Ludwig Schmidt Vaishaal Shankar OOD SSeg VLM 66 1,693 0 13 Feb 2019
Theoretically Principled Trade-off between Robustness and Accuracy Hongyang R. Zhang Yaodong Yu Jiantao Jiao Eric Xing L. Ghaoui Michael I. Jordan 94 2,525 0 24 Jan 2019
ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness Robert Geirhos Patricia Rubisch Claudio Michaelis Matthias Bethge Felix Wichmann Wieland Brendel 72 2,647 0 29 Nov 2018
Excessive Invariance Causes Adversarial Vulnerability J. Jacobsen Jens Behrmann R. Zemel Matthias Bethge AAML 43 166 0 01 Nov 2018
Learning deep representations by mutual information estimation and maximization R. Devon Hjelm A. Fedorov Samuel Lavoie-Marchildon Karan Grewal Phil Bachman Adam Trischler Yoshua Bengio SSL DRL 225 2,649 0 20 Aug 2018
Recognition in Terra Incognita Sara Beery Grant Van Horn Pietro Perona 50 835 0 13 Jul 2018
Representation Learning with Contrastive Predictive Coding Aaron van den Oord Yazhe Li Oriol Vinyals DRL SSL 204 10,152 0 10 Jul 2018