Title
Simplifying DINO via Coding Rate Regularization Ziyang Wu Jingyuan Zhang Druv Pai Xinze Wang Chandan Singh Jianwei Yang Jianfeng Gao Yi-An Ma 501 1 0 17 Feb 2025
Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws Nikhil Sardana Jacob P. Portes Sasha Doubov Jonathan Frankle LRM 322 86 0 31 Dec 2023
A U-turn on Double Descent: Rethinking Parameter Counting in Statistical Learning Alicia Curth Alan Jeffares M. Schaar 43 22 0 29 Oct 2023
Does Double Descent Occur in Self-Supervised Learning? A. Lupidi Yonatan Gideoni Dulhan Jayalath SSL 57 2 0 15 Jul 2023
High-dimensional analysis of double descent for linear regression with random projections Francis R. Bach 70 36 0 02 Mar 2023
Scaling Laws for a Multi-Agent Reinforcement Learning Model Oren Neumann C. Gros 81 27 0 29 Sep 2022
TiCo: Transformation Invariance and Covariance Contrast for Self-Supervised Visual Representation Learning Jiachen Zhu R. M. Moraes Serkan Karakulak Vlad Sobol A. Canziani Yann LeCun SSL 36 22 0 21 Jun 2022
Scaling Laws and Interpretability of Learning from Repeated Data Danny Hernandez Tom B. Brown Tom Conerly Nova Dassarma Dawn Drain ... Catherine Olsson Dario Amodei Nicholas Joseph Jared Kaplan Sam McCandlish 75 117 0 21 May 2022
Training Compute-Optimal Large Language Models Jordan Hoffmann Sebastian Borgeaud A. Mensch Elena Buchatskaya Trevor Cai ... Karen Simonyan Erich Elsen Jack W. Rae Oriol Vinyals Laurent Sifre AI4TS 208 1,976 0 29 Mar 2022
Bias-variance decomposition of overparameterized regression with random linear features J. Rocks Pankaj Mehta 53 12 0 10 Mar 2022
VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning Adrien Bardes Jean Ponce Yann LeCun SSL DML 153 944 0 11 May 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 708 6,127 0 29 Apr 2021
Scaling Scaling Laws with Board Games Andrew Jones 50 43 0 07 Apr 2021
Self-supervised Representation Learning with Relative Predictive Coding Yao-Hung Hubert Tsai Martin Q. Ma Muqiao Yang Han Zhao Louis-Philippe Morency Ruslan Salakhutdinov SSL AI4TS 59 38 0 21 Mar 2021
Barlow Twins: Self-Supervised Learning via Redundancy Reduction Jure Zbontar Li Jing Ishan Misra Yann LeCun Stéphane Deny SSL 344 2,362 0 04 Mar 2021
Scaling Laws for Transfer Danny Hernandez Jared Kaplan T. Henighan Sam McCandlish 90 250 0 02 Feb 2021
Exploring Simple Siamese Representation Learning Xinlei Chen Kaiming He SSL 258 4,072 0 20 Nov 2020
Memorizing without overfitting: Bias, variance, and interpolation in over-parameterized models J. Rocks Pankaj Mehta 92 42 0 26 Oct 2020
Unsupervised Learning of Visual Features by Contrasting Cluster Assignments Mathilde Caron Ishan Misra Julien Mairal Priya Goyal Piotr Bojanowski Armand Joulin OCL SSL 252 4,097 0 17 Jun 2020
Bootstrap your own latent: A new approach to self-supervised Learning Jean-Bastien Grill Florian Strub Florent Altché Corentin Tallec Pierre Harvey Richemond ... M. G. Azar Bilal Piot Koray Kavukcuoglu Rémi Munos Michal Valko SSL 389 6,837 0 13 Jun 2020
On Mutual Information in Contrastive Learning for Visual Representations Mike Wu Chengxu Zhuang Milan Mossé Daniel L. K. Yamins Noah D. Goodman SSL 60 84 0 27 May 2020
Improved Baselines with Momentum Contrastive Learning Xinlei Chen Haoqi Fan Ross B. Girshick Kaiming He SSL 489 3,443 0 09 Mar 2020
A Simple Framework for Contrastive Learning of Visual Representations Ting-Li Chen Simon Kornblith Mohammad Norouzi Geoffrey E. Hinton SSL 375 18,859 0 13 Feb 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 611 4,905 0 23 Jan 2020
Deep Double Descent: Where Bigger Models and More Data Hurt Preetum Nakkiran Gal Kaplun Yamini Bansal Tristan Yang Boaz Barak Ilya Sutskever 121 945 0 04 Dec 2019
Momentum Contrast for Unsupervised Visual Representation Learning Kaiming He Haoqi Fan Yuxin Wu Saining Xie Ross B. Girshick SSL 210 12,121 0 13 Nov 2019
Self-labelling via simultaneous clustering and representation learning Yuki M. Asano Christian Rupprecht Andrea Vedaldi SSL 123 777 0 13 Nov 2019
A Constructive Prediction of the Generalization Error Across Scales Jonathan S. Rosenfeld Amir Rosenfeld Yonatan Belinkov Nir Shavit 105 215 0 27 Sep 2019
Contrastive Multiview Coding Yonglong Tian Dilip Krishnan Phillip Isola SSL 174 2,409 0 13 Jun 2019
Learning Representations by Maximizing Mutual Information Across Views Philip Bachman R. Devon Hjelm William Buchwalter SSL 195 1,477 0 03 Jun 2019
Local Aggregation for Unsupervised Learning of Visual Embeddings Chengxu Zhuang Alex Zhai Daniel L. K. Yamins SSL 95 447 0 29 Mar 2019
Surprises in High-Dimensional Ridgeless Least Squares Interpolation Trevor Hastie Andrea Montanari Saharon Rosset Robert Tibshirani 194 746 0 19 Mar 2019
Reconciling modern machine learning practice and the bias-variance trade-off M. Belkin Daniel J. Hsu Siyuan Ma Soumik Mandal 242 1,655 0 28 Dec 2018
Deep Learning Scaling is Predictable, Empirically Joel Hestness Sharan Narang Newsha Ardalani G. Diamos Heewoo Jun Hassan Kianinejad Md. Mostofa Ali Patwary Yang Yang Yanqi Zhou 110 743 0 01 Dec 2017