Energy-Inspired Self-Supervised Pretraining for Vision Models

2 February 2023

Zicheng Liu

Papers citing "Energy-Inspired Self-Supervised Pretraining for Vision Models"

22 / 22 papers shown

Title
Position Prediction as an Effective Pretraining Strategy Shuangfei Zhai Navdeep Jaitly Jason Ramapuram Dan Busbridge Tatiana Likhomanenko Joseph Y. Cheng Walter A. Talbott Chen Huang Hanlin Goh J. Susskind ViT 66 23 0 15 Jul 2022
Learning Iterative Reasoning through Energy Minimization Yilun Du Shuang Li J. Tenenbaum Igor Mordatch PINN 54 22 0 30 Jun 2022
Masked Siamese Networks for Label-Efficient Learning Mahmoud Assran Mathilde Caron Ishan Misra Piotr Bojanowski Florian Bordes Pascal Vincent Armand Joulin Michael G. Rabbat Nicolas Ballas SSL 67 314 0 14 Apr 2022
Masked Feature Prediction for Self-Supervised Visual Pre-Training Chen Wei Haoqi Fan Saining Xie Chaoxia Wu Alan Yuille Christoph Feichtenhofer ViT 128 661 0 16 Dec 2021
iBOT: Image BERT Pre-Training with Online Tokenizer Jinghao Zhou Chen Wei Huiyu Wang Wei Shen Cihang Xie Alan Yuille Tao Kong 59 722 0 15 Nov 2021
BEiT: BERT Pre-Training of Image Transformers Hangbo Bao Li Dong Songhao Piao Furu Wei ViT 175 2,790 0 15 Jun 2021
VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning Adrien Bardes Jean Ponce Yann LeCun SSL DML 115 917 0 11 May 2021
Barlow Twins: Self-Supervised Learning via Redundancy Reduction Jure Zbontar Li Jing Ishan Misra Yann LeCun Stéphane Deny SSL 223 2,321 0 04 Mar 2021
Zero-Shot Text-to-Image Generation Aditya A. Ramesh Mikhail Pavlov Gabriel Goh Scott Gray Chelsea Voss Alec Radford Mark Chen Ilya Sutskever VLM 327 4,873 0 24 Feb 2021
Understanding self-supervised Learning Dynamics without Contrastive Pairs Yuandong Tian Xinlei Chen Surya Ganguli SSL 181 282 0 12 Feb 2021
Exploring Simple Siamese Representation Learning Xinlei Chen Kaiming He SSL 192 3,992 0 20 Nov 2020
Bootstrap your own latent: A new approach to self-supervised Learning Jean-Bastien Grill Florian Strub Florent Altché Corentin Tallec Pierre Harvey Richemond ... M. G. Azar Bilal Piot Koray Kavukcuoglu Rémi Munos Michal Valko SSL 276 6,718 0 13 Jun 2020
Improved Baselines with Momentum Contrastive Learning Xinlei Chen Haoqi Fan Ross B. Girshick Kaiming He SSL 423 3,397 0 09 Mar 2020
Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One Will Grathwohl Kuan-Chieh Wang J. Jacobsen David Duvenaud Mohammad Norouzi Kevin Swersky VLM 74 536 0 06 Dec 2019
Momentum Contrast for Unsupervised Visual Representation Learning Kaiming He Haoqi Fan Yuxin Wu Saining Xie Ross B. Girshick SSL 113 12,007 0 13 Nov 2019
Generative Modeling by Estimating Gradients of the Data Distribution Yang Song Stefano Ermon SyDa DiffM 178 3,803 0 12 Jul 2019
CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features Sangdoo Yun Dongyoon Han Seong Joon Oh Sanghyuk Chun Junsuk Choe Y. Yoo OOD 581 4,735 0 13 May 2019
On the Anatomy of MCMC-Based Maximum Likelihood Learning of Energy-Based Models Erik Nijkamp Mitch Hill Tian Han Song-Chun Zhu Ying Nian Wu 42 154 0 29 Mar 2019
Energy-based Generative Adversarial Network Jiaqi Zhao Michaël Mathieu Yann LeCun GAN 118 1,112 0 11 Sep 2016
Deep Networks with Stochastic Depth Gao Huang Yu Sun Zhuang Liu Daniel Sedra Kilian Q. Weinberger 151 2,344 0 30 Mar 2016
Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles M. Noroozi Paolo Favaro SSL 135 2,973 0 30 Mar 2016
Rethinking the Inception Architecture for Computer Vision Christian Szegedy Vincent Vanhoucke Sergey Ioffe Jonathon Shlens Z. Wojna 3DV BDL 497 27,231 0 02 Dec 2015