Understanding Deep Contrastive Learning via Coordinate-wise Optimization

29 January 2022

Papers citing "Understanding Deep Contrastive Learning via Coordinate-wise Optimization"

43 / 43 papers shown

Title
Languages Transferred Within the Encoder: On Representation Transfer in Zero-Shot Multilingual Translation Zhi Qu Chenchen Ding Taro Watanabe 105 1 0 12 Jun 2024
Understanding Dimensional Collapse in Contrastive Self-supervised Learning Li Jing Pascal Vincent Yann LeCun Yuandong Tian SSL 106 346 0 18 Oct 2021
Decoupled Contrastive Learning Chun-Hsiao Yeh Cheng-Yao Hong Yen-Chi Hsu Tyng-Luh Liu Yubei Chen Yann LeCun 210 184 0 13 Oct 2021
The Power of Contrast for Feature Learning: A Theoretical Analysis Wenlong Ji Zhun Deng Ryumei Nakada James Zou Linjun Zhang SSL 70 50 0 06 Oct 2021
Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss Jeff Z. HaoChen Colin Wei Adrien Gaidon Tengyu Ma SSL 46 306 0 08 Jun 2021
Toward Understanding the Feature Learning Process of Self-supervised Contrastive Learning Zixin Wen Yuanzhi Li SSL MLT 47 133 0 31 May 2021
Barlow Twins: Self-Supervised Learning via Redundancy Reduction Jure Zbontar Li Jing Ishan Misra Yann LeCun Stéphane Deny SSL 223 2,321 0 04 Mar 2021
Exploring Simple Siamese Representation Learning Xinlei Chen Kaiming He SSL 189 3,992 0 20 Nov 2020
Contrastive Learning with Hard Negative Samples Joshua Robinson Ching-Yao Chuang S. Sra Stefanie Jegelka SSL 109 768 0 09 Oct 2020
Hard Negative Mixing for Contrastive Learning Yannis Kalantidis Mert Bulent Sariyildiz Noé Pion Philippe Weinzaepfel Diane Larlus SSL 93 636 0 02 Oct 2020
Understanding Self-supervised Learning with Dual Deep Networks Yuandong Tian Lantao Yu Xinlei Chen Surya Ganguli SSL 38 79 0 01 Oct 2020
Predicting What You Already Know Helps: Provable Self-Supervised Learning Jason D. Lee Qi Lei Nikunj Saunshi Jiacheng Zhuo SSL 50 187 0 03 Aug 2020
Unsupervised Learning of Visual Features by Contrasting Cluster Assignments Mathilde Caron Ishan Misra Julien Mairal Priya Goyal Piotr Bojanowski Armand Joulin OCL SSL 159 4,051 0 17 Jun 2020
Bootstrap your own latent: A new approach to self-supervised Learning Jean-Bastien Grill Florian Strub Florent Altché Corentin Tallec Pierre Harvey Richemond ... M. G. Azar Bilal Piot Koray Kavukcuoglu Rémi Munos Michal Valko SSL 276 6,718 0 13 Jun 2020
What Makes for Good Views for Contrastive Learning? Yonglong Tian Chen Sun Ben Poole Dilip Krishnan Cordelia Schmid Phillip Isola SSL 75 1,313 0 20 May 2020
Supervised Contrastive Learning Prannay Khosla Piotr Teterwak Chen Wang Aaron Sarna Yonglong Tian Phillip Isola Aaron Maschinot Ce Liu Dilip Krishnan SSL 120 4,505 0 23 Apr 2020
A Comparison of Metric Learning Loss Functions for End-To-End Speaker Verification Juan Manuel Coria H. Bredin Sahar Ghannay S. Rosset 38 15 0 31 Mar 2020
A Simple Framework for Contrastive Learning of Visual Representations Ting-Li Chen Simon Kornblith Mohammad Norouzi Geoffrey E. Hinton SSL 255 18,607 0 13 Feb 2020
On Layer Normalization in the Transformer Architecture Ruibin Xiong Yunchang Yang Di He Kai Zheng Shuxin Zheng Chen Xing Huishuai Zhang Yanyan Lan Liwei Wang Tie-Yan Liu AI4CE 95 973 0 12 Feb 2020
Self-Supervised Learning of Pretext-Invariant Representations Ishan Misra Laurens van der Maaten SSL VLM 74 1,448 0 04 Dec 2019
Momentum Contrast for Unsupervised Visual Representation Learning Kaiming He Haoqi Fan Yuxin Wu Saining Xie Ross B. Girshick SSL 113 12,007 0 13 Nov 2019
Contrastive Multiview Coding Yonglong Tian Dilip Krishnan Phillip Isola SSL 131 2,385 0 13 Jun 2019
A Theoretical Analysis of Contrastive Unsupervised Representation Learning Sanjeev Arora H. Khandeparkar M. Khodak Orestis Plevrakis Nikunj Saunshi SSL 86 768 0 25 Feb 2019
Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers Zeyuan Allen-Zhu Yuanzhi Li Yingyu Liang MLT 131 769 0 12 Nov 2018
A theoretical framework for deep locally connected ReLU network Yuandong Tian PINN 41 10 0 28 Sep 2018
Deep Clustering for Unsupervised Learning of Visual Features Mathilde Caron Piotr Bojanowski Armand Joulin Matthijs Douze SSL 78 1,878 0 15 Jul 2018
Representation Learning with Contrastive Predictive Coding Aaron van den Oord Yazhe Li Oriol Vinyals DRL SSL 229 10,152 0 10 Jul 2018
Algorithmic Regularization in Learning Deep Homogeneous Models: Layers are Automatically Balanced S. Du Wei Hu Jason D. Lee MLT 119 239 0 04 Jun 2018
Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination Zhirong Wu Yuanjun Xiong Stella X. Yu Dahua Lin SSL 148 3,437 0 05 May 2018
On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization Sanjeev Arora Nadav Cohen Elad Hazan 88 481 0 19 Feb 2018
MINE: Mutual Information Neural Estimation Mohamed Ishmael Belghazi A. Baratin Sai Rajeswar Sherjil Ozair Yoshua Bengio Aaron Courville R. Devon Hjelm DRL 143 1,264 0 12 Jan 2018
Spurious Local Minima are Common in Two-Layer ReLU Neural Networks Itay Safran Ohad Shamir 124 263 0 24 Dec 2017
Deep linear neural networks with arbitrary loss: All local minima are global T. Laurent J. V. Brecht ODL 34 136 0 05 Dec 2017
Gradient Descent Learns One-hidden-layer CNN: Don't be Afraid of Spurious Local Minima S. Du Jason D. Lee Yuandong Tian Barnabás Póczós Aarti Singh MLT 125 236 0 03 Dec 2017
Identity Matters in Deep Learning Moritz Hardt Tengyu Ma OOD 61 399 0 14 Nov 2016
Densely Connected Convolutional Networks Gao Huang Zhuang Liu Laurens van der Maaten Kilian Q. Weinberger PINN 3DV 631 36,599 0 25 Aug 2016
Layer Normalization Jimmy Lei Ba J. Kiros Geoffrey E. Hinton 251 10,412 0 21 Jul 2016
Deep Learning without Poor Local Minima Kenji Kawaguchi ODL 165 922 0 23 May 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 1.4K 192,638 0 10 Dec 2015
Deep Metric Learning via Lifted Structured Feature Embedding Hyun Oh Song Yu Xiang Stefanie Jegelka Silvio Savarese FedML SSL DML 75 1,641 0 19 Nov 2015
FaceNet: A Unified Embedding for Face Recognition and Clustering Florian Schroff Dmitry Kalenichenko James Philbin 3DH 281 13,079 0 12 Mar 2015
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 842 149,474 0 22 Dec 2014
Exact solutions to the nonlinear dynamics of learning in deep linear neural networks Andrew M. Saxe James L. McClelland Surya Ganguli ODL 125 1,830 0 20 Dec 2013