Random Teachers are Good Teachers

23 February 2023

Papers citing "Random Teachers are Good Teachers"

24 / 24 papers shown

Title
Provable Weak-to-Strong Generalization via Benign Overfitting David X. Wu A. Sahai 116 10 0 06 Oct 2024
PhiNets: Brain-inspired Non-contrastive Learning Based on Temporal Prediction Hypothesis Satoki Ishikawa Makoto Yamada Han Bao Yuki Takezawa 118 0 0 23 May 2024
The Curious Case of Benign Memorization Sotiris Anagnostidis Gregor Bachmann Lorenzo Noci Thomas Hofmann AAML 85 10 0 25 Oct 2022
When Does Re-initialization Work? Sheheryar Zaidi Tudor Berariu Hyunjik Kim J. Bornschein Claudia Clopath Yee Whye Teh Razvan Pascanu 61 10 0 20 Jun 2022
Masked Siamese Networks for Label-Efficient Learning Mahmoud Assran Mathilde Caron Ishan Misra Piotr Bojanowski Florian Bordes Pascal Vincent Armand Joulin Michael G. Rabbat Nicolas Ballas SSL 75 318 0 14 Apr 2022
How Does SimSiam Avoid Collapse Without Negative Samples? A Unified Understanding with Self-supervised Contrastive Learning Chaoning Zhang Kang Zhang Chenshuang Zhang T. Pham Chang D. Yoo In So Kweon SSL 52 71 0 30 Mar 2022
Does Knowledge Distillation Really Work? Samuel Stanton Pavel Izmailov Polina Kirichenko Alexander A. Alemi A. Wilson FedML 57 220 0 10 Jun 2021
Knowledge distillation: A good teacher is patient and consistent Lucas Beyer Xiaohua Zhai Amelie Royer L. Markeeva Rohan Anil Alexander Kolesnikov VLM 94 293 0 09 Jun 2021
Towards Understanding Knowledge Distillation Mary Phuong Christoph H. Lampert 63 316 0 27 May 2021
VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning Adrien Bardes Jean Ponce Yann LeCun SSL DML 146 931 0 11 May 2021
What Are Bayesian Neural Network Posteriors Really Like? Pavel Izmailov Sharad Vikram Matthew D. Hoffman A. Wilson UQCV BDL 65 382 0 29 Apr 2021
Barlow Twins: Self-Supervised Learning via Redundancy Reduction Jure Zbontar Li Jing Ishan Misra Yann LeCun Stéphane Deny SSL 270 2,338 0 04 Mar 2021
Understanding self-supervised Learning Dynamics without Contrastive Pairs Yuandong Tian Xinlei Chen Surya Ganguli SSL 196 281 0 12 Feb 2021
Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning Zeyuan Allen-Zhu Yuanzhi Li FedML 114 370 0 17 Dec 2020
Exploring Simple Siamese Representation Learning Xinlei Chen Kaiming He SSL 242 4,036 0 20 Nov 2020
Knowledge Distillation in Wide Neural Networks: Risk Bound, Data Efficiency and Imperfect Teacher Guangda Ji Zhanxing Zhu 71 44 0 20 Oct 2020
Bootstrap your own latent: A new approach to self-supervised Learning Jean-Bastien Grill Florian Strub Florent Altché Corentin Tallec Pierre Harvey Richemond ... M. G. Azar Bilal Piot Koray Kavukcuoglu Rémi Munos Michal Valko SSL 325 6,773 0 13 Jun 2020
Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation Linfeng Zhang Jiebo Song Anni Gao Jingwei Chen Chenglong Bao Kaisheng Ma FedML 60 857 0 17 May 2019
Asymmetric Valleys: Beyond Sharp and Flat Local Minima Haowei He Gao Huang Yang Yuan ODL MLT 61 150 0 02 Feb 2019
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks Jonathan Frankle Michael Carbin 198 3,457 0 09 Mar 2018
Essentially No Barriers in Neural Network Energy Landscape Felix Dräxler K. Veschgini M. Salmhofer Fred Hamprecht MoMe 105 432 0 02 Mar 2018
FaceNet: A Unified Embedding for Face Recognition and Clustering Florian Schroff Dmitry Kalenichenko James Philbin 3DH 328 13,123 0 12 Mar 2015
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification Kaiming He Xinming Zhang Shaoqing Ren Jian Sun VLM 270 18,587 0 06 Feb 2015
Do Deep Nets Really Need to be Deep? Lei Jimmy Ba R. Caruana 158 2,117 0 21 Dec 2013