Understanding Contrastive Learning Requires Incorporating Inductive Biases

28 February 2022

Papers citing "Understanding Contrastive Learning Requires Incorporating Inductive Biases"

21 / 71 papers shown

Title
Understanding Multimodal Contrastive Learning and Incorporating Unpaired Data Ryumei Nakada Halil Ibrahim Gulluk Zhun Deng Wenlong Ji James Zou Linjun Zhang SSL VLM 106 41 0 13 Feb 2023
Evaluating Self-Supervised Learning via Risk Decomposition Yann Dubois Tatsunori Hashimoto Percy Liang 71 9 0 06 Feb 2023
The SSL Interplay: Augmentations, Inductive Bias, and Generalization Vivien A. Cabannes B. Kiani Randall Balestriero Yann LeCun A. Bietti SSL 92 33 0 06 Feb 2023
Deciphering the Projection Head: Representation Evaluation Self-supervised Learning Jiajun Ma Tianyang Hu Wei Cao 76 8 0 28 Jan 2023
Understanding Self-Supervised Pretraining with Part-Aware Representation Learning Jie Zhu Jiyang Qi Mingyu Ding Xiaokang Chen Ping Luo Xinggang Wang Wenyu Liu Leye Wang Jingdong Wang SSL 104 8 0 27 Jan 2023
GEDI: GEnerative and DIscriminative Training for Self-Supervised Learning Emanuele Sansone Robin Manhaeve SSL 105 9 0 27 Dec 2022
A Theoretical Study of Inductive Biases in Contrastive Learning Jeff Z. HaoChen Tengyu Ma UQCV SSL 119 34 0 27 Nov 2022
ProtoX: Explaining a Reinforcement Learning Agent via Prototyping Ronilo Ragodos Tong Wang Qihang Lin Xun Zhou 65 7 0 06 Nov 2022
Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models Hong Liu Sang Michael Xie Zhiyuan Li Tengyu Ma AI4CE 135 55 0 25 Oct 2022
The Curious Case of Benign Memorization Sotiris Anagnostidis Gregor Bachmann Lorenzo Noci Thomas Hofmann AAML 132 10 0 25 Oct 2022
A Kernel-Based View of Language Model Fine-Tuning Sadhika Malladi Alexander Wettig Dingli Yu Danqi Chen Sanjeev Arora VLM 157 69 0 11 Oct 2022
Contrastive Learning Can Find An Optimal Basis For Approximately View-Invariant Functions Daniel D. Johnson Ayoub El Hanchi Chris J. Maddison SSL 98 24 0 04 Oct 2022
What shapes the loss landscape of self-supervised learning? Liu Ziyin Ekdeep Singh Lubana Masakuni Ueda Hidenori Tanaka 105 21 0 02 Oct 2022
Improving Self-Supervised Learning by Characterizing Idealized Representations Yann Dubois Tatsunori Hashimoto Stefano Ermon Percy Liang SSL 167 43 0 13 Sep 2022
Analyzing Data-Centric Properties for Graph Contrastive Learning Puja Trivedi Ekdeep Singh Lubana Mark Heimann Danai Koutra Jayaraman J. Thiagarajan 103 11 0 04 Aug 2022
Integrating Prior Knowledge in Contrastive Learning with Kernel Benoit Dufumier C. Barbano Robin Louiset Edouard Duchesnay Pietro Gori SSL 71 8 0 03 Jun 2022
Understanding the Role of Nonlinearity in Training Dynamics of Contrastive Learning Yuandong Tian MLT 125 14 0 02 Jun 2022
Your Contrastive Learning Is Secretly Doing Stochastic Neighbor Embedding Tianyang Hu Zhili Liu Fengwei Zhou Wei Cao Weiran Huang SSL 105 28 0 30 May 2022
Orchestra: Unsupervised Federated Learning via Globally Consistent Clustering Ekdeep Singh Lubana Chi Ian Tang F. Kawsar Robert P. Dick Akhil Mathur FedML 75 54 0 23 May 2022
The Mechanism of Prediction Head in Non-contrastive Self-supervised Learning Zixin Wen Yuanzhi Li SSL 116 35 0 12 May 2022
Do More Negative Samples Necessarily Hurt in Contrastive Learning? Pranjal Awasthi Nishanth Dikkala Pritish Kamath 99 43 0 03 May 2022