Position: An Empirically Grounded Identifiability Theory Will Accelerate Self-Supervised Learning Research

17 April 2025

Papers citing "Position: An Empirically Grounded Identifiability Theory Will Accelerate Self-Supervised Learning Research"

39 / 89 papers shown

Title
Understanding the Role of Nonlinearity in Training Dynamics of Contrastive Learning Yuandong Tian MLT 50 12 0 02 Jun 2022
Contrastive and Non-Contrastive Self-Supervised Learning Recover Global and Local Spectral Embedding Methods Randall Balestriero Yann LeCun SSL 73 130 0 23 May 2022
Learnable latent embeddings for joint behavioral and neural analysis Steffen Schneider Jin Hwa Lee Mackenzie W. Mathis 62 224 0 01 Apr 2022
Weakly supervised causal representation learning Johann Brehmer P. D. Haan Phillip Lippe Taco S. Cohen OOD CML 57 129 0 30 Mar 2022
End-to-end Algorithm Synthesis with Recurrent Networks: Logical Extrapolation Without Overthinking Arpit Bansal Avi Schwarzschild Eitan Borgnia Z. Emam Furong Huang Micah Goldblum Tom Goldstein LRM 36 24 0 11 Feb 2022
Omnivore: A Single Model for Many Visual Modalities Rohit Girdhar Mannat Singh Nikhil Ravi Laurens van der Maaten Armand Joulin Ishan Misra 253 233 0 20 Jan 2022
Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets Alethea Power Yuri Burda Harrison Edwards Igor Babuschkin Vedant Misra 73 354 0 06 Jan 2022
An Explanation of In-context Learning as Implicit Bayesian Inference Sang Michael Xie Aditi Raghunathan Percy Liang Tengyu Ma ReLM BDL VPVLM LRM 175 746 0 03 Nov 2021
Understanding Dimensional Collapse in Contrastive Self-supervised Learning Li Jing Pascal Vincent Yann LeCun Yuandong Tian SSL 111 349 0 18 Oct 2021
Visual Representation Learning Does Not Generalize Strongly Within the Same Domain Lukas Schott Julius von Kügelgen Frederik Trauble Peter V. Gehler Chris Russell Matthias Bethge Bernhard Schölkopf Francesco Locatello Wieland Brendel OOD DRL 87 72 0 17 Jul 2021
Understanding Latent Correlation-Based Multiview Learning and Self-Supervision: An Identifiability Perspective Qinjie Lyu Xiao Fu Weiran Wang Songtao Lu SSL 44 31 0 14 Jun 2021
Independent mechanism analysis, a new concept? Luigi Gresele Julius von Kügelgen Vincent Stimper Bernhard Schölkopf M. Besserve CML 45 102 0 09 Jun 2021
Scaling Vision Transformers Xiaohua Zhai Alexander Kolesnikov N. Houlsby Lucas Beyer ViT 128 1,080 0 08 Jun 2021
Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges M. Bronstein Joan Bruna Taco S. Cohen Petar Velivcković GNN 327 1,147 0 27 Apr 2021
Barlow Twins: Self-Supervised Learning via Redundancy Reduction Jure Zbontar Li Jing Ishan Misra Yann LeCun Stéphane Deny SSL 285 2,338 0 04 Mar 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 816 29,167 0 26 Feb 2021
Towards Causal Representation Learning Bernhard Schölkopf Francesco Locatello Stefan Bauer Nan Rosemary Ke Nal Kalchbrenner Anirudh Goyal Yoshua Bengio OOD CML AI4CE 109 322 0 22 Feb 2021
Contrastive Learning Inverts the Data Generating Process Roland S. Zimmermann Yash Sharma Steffen Schneider Matthias Bethge Wieland Brendel SSL 264 218 0 17 Feb 2021
Scaling Laws for Transfer Danny Hernandez Jared Kaplan T. Henighan Sam McCandlish 69 244 0 02 Feb 2021
Learning identifiable and interpretable latent models of high-dimensional neural activity using pi-VAE Ding Zhou Xue-Xin Wei DRL 199 81 0 09 Nov 2020
Scaling Laws for Autoregressive Generative Modeling T. Henighan Jared Kaplan Mor Katz Mark Chen Christopher Hesse ... Nick Ryder Daniel M. Ziegler John Schulman Dario Amodei Sam McCandlish 87 419 0 28 Oct 2020
Prevalence of Neural Collapse during the terminal phase of deep learning training Vardan Papyan Xuemei Han D. Donoho 174 572 0 18 Aug 2020
Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Coding David A. Klindt Lukas Schott Yash Sharma Ivan Ustyuzhaninov Wieland Brendel Matthias Bethge Dylan M. Paiton CML 84 132 0 21 Jul 2020
Independent Innovation Analysis for Nonlinear Vector Autoregressive Process H. Morioka Hermanni Hälvä Aapo Hyvarinen 82 22 0 19 Jun 2020
Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere Tongzhou Wang Phillip Isola SSL 138 1,826 0 20 May 2020
ICE-BeeM: Identifiable Conditional Energy-Based Deep Models Based on Nonlinear ICA Ilyes Khemakhem R. Monti Diederik P. Kingma Aapo Hyvarinen CML 64 114 0 26 Feb 2020
A Simple Framework for Contrastive Learning of Visual Representations Ting-Li Chen Simon Kornblith Mohammad Norouzi Geoffrey E. Hinton SSL 331 18,721 0 13 Feb 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 532 4,773 0 23 Jan 2020
Variational Autoencoders and Nonlinear ICA: A Unifying Framework Ilyes Khemakhem Diederik P. Kingma Ricardo Pio Monti Aapo Hyvarinen OOD 64 593 0 10 Jul 2019
Gradient-Based Neural DAG Learning Sébastien Lachapelle P. Brouillard T. Deleu Simon Lacoste-Julien BDL CML 50 273 0 05 Jun 2019
Unsupervised Model Selection for Variational Disentangled Representation Learning Sunny Duan Loic Matthey Andre Saraiva Nicholas Watters Christopher P. Burgess Alexander Lerchner I. Higgins OOD DRL 66 80 0 29 May 2019
SGD on Neural Networks Learns Functions of Increasing Complexity Preetum Nakkiran Gal Kaplun Dimitris Kalimeris Tristan Yang Benjamin L. Edelman Fred Zhang Boaz Barak MLT 128 247 0 28 May 2019
Towards a Definition of Disentangled Representations I. Higgins David Amos David Pfau S. Racanière Loic Matthey Danilo Jimenez Rezende Alexander Lerchner OCL DRL 95 480 0 05 Dec 2018
Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations Francesco Locatello Stefan Bauer Mario Lucic Gunnar Rätsch Sylvain Gelly Bernhard Schölkopf Olivier Bachem OOD 113 1,466 0 29 Nov 2018
Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learning Aapo Hyvarinen Hiroaki Sasaki Richard Turner OOD CML 92 327 0 22 May 2018
Emergence of Invariance and Disentanglement in Deep Representations Alessandro Achille Stefano Soatto OOD DRL 88 476 0 05 Jun 2017
Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA Aapo Hyvarinen H. Morioka CML OOD AI4TS 58 409 0 20 May 2016
Deep Convolutional Inverse Graphics Network Tejas D. Kulkarni William F. Whitney Pushmeet Kohli J. Tenenbaum DRL BDL 94 929 0 11 Mar 2015
Distributed Representations of Words and Phrases and their Compositionality Tomas Mikolov Ilya Sutskever Kai Chen G. Corrado J. Dean NAI OCL 361 33,500 0 16 Oct 2013