v1v2 (latest)

On the Stepwise Nature of Self-Supervised Learning

27 March 2023

Papers citing "On the Stepwise Nature of Self-Supervised Learning"

26 / 26 papers shown

Title
Alternating Gradient Flows: A Theory of Feature Learning in Two-layer Neural Networks D. Kunin Giovanni Luca Marchetti F. Chen Dhruva Karkada James B. Simon M. DeWeese Surya Ganguli Nina Miolane 28 0 0 06 Jun 2025
Saddle-To-Saddle Dynamics in Deep ReLU Networks: Low-Rank Bias in the First Saddle Escape Ioannis Bantzis James B. Simon Arthur Jacot ODL 51 0 0 27 May 2025
AdaDim: Dimensionality Adaptation for SSL Representational Dynamics Kiran Kokilepersaud Mohit Prabhushankar Ghassan AlRegib 94 0 0 18 May 2025
Position: An Empirically Grounded Identifiability Theory Will Accelerate Self-Supervised Learning Research Patrik Reizinger Randall Balestriero David Klindt Wieland Brendel 199 1 0 17 Apr 2025
Position: Solve Layerwise Linear Models First to Understand Neural Dynamical Phenomena (Neural Collapse, Emergence, Lazy/Rich Regime, and Grokking) Yoonsoo Nam Seok Hyeong Lee Clementine Domine Yea Chan Park Charles London Wonyl Choi Niclas Goring Seungjai Lee AI4CE 209 1 0 28 Feb 2025
It's Not Just a Phase: On Investigating Phase Transitions in Deep Learning-based Side-channel Analysis Sengim Karayalçin Marina Krček Stjepan Picek AAML 159 0 0 01 Feb 2025
Infinite Width Limits of Self Supervised Neural Networks Maximilian Fleissner Gautham Govind Anil Debarghya Ghoshdastidar SSL 465 1 0 17 Nov 2024
Cross-Entropy Is All You Need To Invert the Data Generating Process Patrik Reizinger Alice Bizeul Attila Juhos Julia E. Vogt Randall Balestriero Wieland Brendel David Klindt SSL OOD BDL DRL 229 6 0 29 Oct 2024
The Optimization Landscape of SGD Across the Feature Learning Strength Alexander B. Atanasov Alexandru Meterez James B. Simon Cengiz Pehlevan 150 6 0 06 Oct 2024
Towards Automatic Assessment of Self-Supervised Speech Models using Rank Zakaria Aldeneh Vimal Thilak Takuya Higuchi B. Theobald Tatiana Likhomanenko SSL 153 1 0 16 Sep 2024
How JEPA Avoids Noisy Features: The Implicit Bias of Deep Linear Self Distillation Networks Etai Littwin Omid Saremi Madhu Advani Vimal Thilak Preetum Nakkiran Chen Huang Joshua Susskind 78 5 0 03 Jul 2024
InfoNCE: Identifying the Gap Between Theory and Practice E. Rusak Patrik Reizinger Attila Juhos Oliver Bringmann Roland S. Zimmermann Wieland Brendel 130 11 0 28 Jun 2024
Training Dynamics of Nonlinear Contrastive Learning Model in the High Dimensional Limit Lineghuan Meng Chuang Wang 53 1 0 11 Jun 2024
Phase Transitions in the Output Distribution of Large Language Models Julian Arnold Flemming Holtorf Frank Schafer Niels Lörch 69 2 0 27 May 2024
Mechanistic Interpretability for AI Safety -- A Review Leonard Bereska E. Gavves AI4CE 135 158 0 22 Apr 2024
Auxiliary task demands mask the capabilities of smaller language models Jennifer Hu Michael C. Frank ELM 94 32 0 03 Apr 2024
When can we Approximate Wide Contrastive Models with Neural Tangent Kernels and Principal Component Analysis? Gautham Govind Anil Pascal Esser Debarghya Ghoshdastidar 81 1 0 13 Mar 2024
Compression of Structured Data with Autoencoders: Provable Benefit of Nonlinearities and Depth Kevin Kögler Aleksandr Shevchenko Hamed Hassani Marco Mondelli MLT 85 1 0 07 Feb 2024
FroSSL: Frobenius Norm Minimization for Efficient Multiview Self-Supervised Learning Oscar Skean Aayush Dhakal Nathan Jacobs Luis Gonzalo Sánchez Giraldo 70 0 0 04 Oct 2023
Representation Learning Dynamics of Self-Supervised Models Pascal Esser Satyaki Mukherjee Debarghya Ghoshdastidar SSL 82 3 0 05 Sep 2023
Transformers learn through gradual rank increase Enric Boix-Adserà Etai Littwin Emmanuel Abbe Samy Bengio J. Susskind 102 37 0 12 Jun 2023
The Quantization Model of Neural Scaling Eric J. Michaud Ziming Liu Uzay Girit Max Tegmark MILM 125 89 0 23 Mar 2023
The SSL Interplay: Augmentations, Inductive Bias, and Generalization Vivien A. Cabannes B. Kiani Randall Balestriero Yann LeCun A. Bietti SSL 92 33 0 06 Feb 2023
Neural networks learn to magnify areas near decision boundaries Jacob A. Zavatone-Veth Sheng Yang Julian Rubinfien Cengiz Pehlevan MLT AI4CE 72 6 0 26 Jan 2023
Implicit variance regularization in non-contrastive SSL Manu Srinath Halvagal Axel Laborieux Friedemann Zenke 126 10 0 09 Dec 2022
Unsupervised Learning of Visual Features by Contrasting Cluster Assignments Mathilde Caron Ishan Misra Julien Mairal Priya Goyal Piotr Bojanowski Armand Joulin OCL SSL 342 4,109 0 17 Jun 2020