v1v2 (latest)

Improving Self-Supervised Learning by Characterizing Idealized Representations

13 September 2022

Tatsunori Hashimoto

Papers citing "Improving Self-Supervised Learning by Characterizing Idealized Representations"

50 / 85 papers shown

Title
Can Masked Autoencoders Also Listen to Birds? Lukas Rauch Ilyass Moummad René Heinrich Alexis Joly Bernhard Sick Christoph Scholz 121 0 0 17 Apr 2025
Locality Alignment Improves Vision-Language Models Ian Covert Tony Sun James Zou Tatsunori Hashimoto VLM 239 6 0 14 Oct 2024
InfoNCE: Identifying the Gap Between Theory and Practice E. Rusak Patrik Reizinger Attila Juhos Oliver Bringmann Roland S. Zimmermann Wieland Brendel 99 10 0 28 Jun 2024
Do More Negative Samples Necessarily Hurt in Contrastive Learning? Pranjal Awasthi Nishanth Dikkala Pritish Kamath 97 43 0 03 May 2022
How Does SimSiam Avoid Collapse Without Negative Samples? A Unified Understanding with Self-supervised Contrastive Learning Chaoning Zhang Kang Zhang Chenshuang Zhang T. Pham Chang D. Yoo In So Kweon SSL 75 74 0 30 Mar 2022
Contrasting the landscape of contrastive and non-contrastive learning Ashwini Pokle Jinjin Tian Yuchen Li Andrej Risteski SSL 83 28 0 29 Mar 2022
Chaos is a Ladder: A New Theoretical Understanding of Contrastive Learning via Augmentation Overlap Yifei Wang Qi Zhang Yisen Wang Jiansheng Yang Zhouchen Lin 68 102 0 25 Mar 2022
Understanding Contrastive Learning Requires Incorporating Inductive Biases Nikunj Saunshi Jordan T. Ash Surbhi Goel Dipendra Kumar Misra Cyril Zhang Sanjeev Arora Sham Kakade A. Krishnamurthy SSL 84 113 0 28 Feb 2022
Limitations of Neural Collapse for Understanding Generalization in Deep Learning Like Hui M. Belkin Preetum Nakkiran AI4CE 44 56 0 17 Feb 2022
Optimal Representations for Covariate Shift Yangjun Ruan Yann Dubois Chris J. Maddison OOD 97 69 0 31 Dec 2021
On the Role of Neural Collapse in Transfer Learning Tomer Galanti András Gyorgy Marcus Hutter SSL 74 93 0 30 Dec 2021
Understanding Dimensional Collapse in Contrastive Self-supervised Learning Li Jing Pascal Vincent Yann LeCun Yuandong Tian SSL 120 358 0 18 Oct 2021
An Unconstrained Layer-Peeled Perspective on Neural Collapse Wenlong Ji Yiping Lu Yiliang Zhang Zhun Deng Weijie J. Su 178 87 0 06 Oct 2021
Lossy Compression for Lossless Prediction Yann Dubois Benjamin Bloem-Reddy Karen Ullrich Chris J. Maddison 93 62 0 21 Jun 2021
Investigating the Role of Negatives in Contrastive Representation Learning Jordan T. Ash Surbhi Goel A. Krishnamurthy Dipendra Kumar Misra SSL 72 53 0 18 Jun 2021
Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style Julius von Kügelgen Yash Sharma Luigi Gresele Wieland Brendel Bernhard Schölkopf M. Besserve Francesco Locatello 99 317 0 08 Jun 2021
Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss Jeff Z. HaoChen Colin Wei Adrien Gaidon Tengyu Ma SSL 77 321 0 08 Jun 2021
Toward Understanding the Feature Learning Process of Self-supervised Contrastive Learning Zixin Wen Yuanzhi Li SSL MLT 74 136 0 31 May 2021
VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning Adrien Bardes Jean Ponce Yann LeCun SSL DML 153 944 0 11 May 2021
A Geometric Analysis of Neural Collapse with Unconstrained Features Zhihui Zhu Tianyu Ding Jinxin Zhou Xiao Li Chong You Jeremias Sulam Qing Qu 69 205 0 06 May 2021
On Feature Decorrelation in Self-Supervised Learning Tianyu Hua Wenxiao Wang Zihui Xue Sucheng Ren Yue Wang Hang Zhao SSL OOD 183 195 0 02 May 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 706 6,121 0 29 Apr 2021
Barlow Twins: Self-Supervised Learning via Redundancy Reduction Jure Zbontar Li Jing Ishan Misra Yann LeCun Stéphane Deny SSL 344 2,362 0 04 Mar 2021
Self-supervised Pretraining of Visual Features in the Wild Priya Goyal Mathilde Caron Benjamin Lefaudeux Min Xu Pengchao Wang ... Mannat Singh Vitaliy Liptchinsky Ishan Misra Armand Joulin Piotr Bojanowski VLM SSL 76 274 0 02 Mar 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 967 29,810 0 26 Feb 2021
Dissecting Supervised Contrastive Learning Florian Graf Christoph Hofer Marc Niethammer Roland Kwitt SSL 156 74 0 17 Feb 2021
Understanding Negative Samples in Instance Discriminative Self-supervised Representation Learning Kento Nozawa Issei Sato SSL 113 46 0 13 Feb 2021
Understanding self-supervised Learning Dynamics without Contrastive Pairs Yuandong Tian Xinlei Chen Surya Ganguli SSL 213 286 0 12 Feb 2021
Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Training Cong Fang Hangfeng He Qi Long Weijie J. Su FAtt 170 172 0 29 Jan 2021
SEED: Self-supervised Distillation For Visual Representation Zhiyuan Fang Jianfeng Wang Lijuan Wang Lei Zhang Yezhou Yang Zicheng Liu SSL 301 194 0 12 Jan 2021
Minimum Excess Risk in Bayesian Learning Aolin Xu Maxim Raginsky 412 40 0 29 Dec 2020
Understanding the Behaviour of Contrastive Loss Feng Wang Huaping Liu SSL 105 692 0 15 Dec 2020
On the emergence of simplex symmetry in the final and penultimate layers of neural network classifiers E. Weinan Stephan Wojtowytsch 76 45 0 10 Dec 2020
Neural collapse with unconstrained features D. Mixon Hans Parshall Jianzong Pi 72 121 0 23 Nov 2020
Exploring Simple Siamese Representation Learning Xinlei Chen Kaiming He SSL 258 4,072 0 20 Nov 2020
Intriguing Properties of Contrastive Losses Ting Chen Calvin Luo Lala Li 69 177 0 05 Nov 2020
Why Do Better Loss Functions Lead to Less Transferable Features? Simon Kornblith Ting-Li Chen Honglak Lee Mohammad Norouzi FaML 91 92 0 30 Oct 2020
BYOL works even without batch statistics Pierre Harvey Richemond Jean-Bastien Grill Florent Altché Corentin Tallec Florian Strub ... Samuel L. Smith Soham De Razvan Pascanu Bilal Piot Michal Valko SSL 295 115 0 20 Oct 2020
Improving Transformation Invariance in Contrastive Representation Learning Adam Foster Rattana Pukdee Tom Rainforth 96 23 0 19 Oct 2020
For self-supervised learning, Rationality implies generalization, provably Yamini Bansal Gal Kaplun Boaz Barak OOD SSL 107 22 0 16 Oct 2020
Representation Learning via Invariant Causal Mechanisms Jovana Mitrović Brian McWilliams Jacob Walker Lars Buesing Charles Blundell CML OOD SSL 63 250 0 15 Oct 2020
Understanding Self-supervised Learning with Dual Deep Networks Yuandong Tian Lantao Yu Xinlei Chen Surya Ganguli SSL 64 80 0 01 Oct 2020
Learning Optimal Representations with the Decodable Information Bottleneck Yann Dubois Douwe Kiela D. Schwab Ramakrishna Vedantam 82 43 0 27 Sep 2020
Contrastive learning, multi-view redundancy, and linear models Christopher Tosh A. Krishnamurthy Daniel J. Hsu SSL 82 167 0 24 Aug 2020
Prevalence of Neural Collapse during the terminal phase of deep learning training Vardan Papyan Xuemei Han D. Donoho 205 582 0 18 Aug 2020
Predicting What You Already Know Helps: Provable Self-Supervised Learning Jason D. Lee Qi Lei Nikunj Saunshi Jiacheng Zhuo SSL 79 189 0 03 Aug 2020
Whitening for Self-Supervised Representation Learning Aleksandr Ermolov Aliaksandr Siarohin E. Sangineto N. Sebe SSL 93 315 0 13 Jul 2020
Unsupervised Learning of Visual Features by Contrasting Cluster Assignments Mathilde Caron Ishan Misra Julien Mairal Priya Goyal Piotr Bojanowski Armand Joulin OCL SSL 252 4,097 0 17 Jun 2020
Bootstrap your own latent: A new approach to self-supervised Learning Jean-Bastien Grill Florian Strub Florent Altché Corentin Tallec Pierre Harvey Richemond ... M. G. Azar Bilal Piot Koray Kavukcuoglu Rémi Munos Michal Valko SSL 389 6,833 0 13 Jun 2020
On Mutual Information in Contrastive Learning for Visual Representations Mike Wu Chengxu Zhuang Milan Mossé Daniel L. K. Yamins Noah D. Goodman SSL 60 84 0 27 May 2020