v1v2 (latest)

MimCo: Masked Image Modeling Pre-training with Contrastive Teacher

7 September 2022

Hao Li

Papers citing "MimCo: Masked Image Modeling Pre-training with Contrastive Teacher"

42 / 42 papers shown

Title
A Contrastive Learning Foundation Model Based on Perfectly Aligned Sample Pairs for Remote Sensing Images Hengtong Shen Haiyan Gu Haitao Li Yi Yang Agen qiu SSL 131 0 0 26 May 2025
RingMo-Aerial: An Aerial Remote Sensing Foundation Model With Affine Transformation Contrastive Learning Wenhui Diao Haichen Yu Kaiyue Kang Tong Ling Di Liu ... Hanbo Bi Libo Ren Xuexue Li Yongqiang Mao Xian Sun 240 1 0 20 Sep 2024
Masked Feature Prediction for Self-Supervised Visual Pre-Training Chen Wei Haoqi Fan Saining Xie Chaoxia Wu Alan Yuille Christoph Feichtenhofer ViT 145 668 0 16 Dec 2021
PeCo: Perceptual Codebook for BERT Pre-training of Vision Transformers Xiaoyi Dong Jianmin Bao Ting Zhang Dongdong Chen Weiming Zhang Lu Yuan Dong Chen Fang Wen Nenghai Yu Baining Guo ViT 103 243 0 24 Nov 2021
SimMIM: A Simple Framework for Masked Image Modeling Zhenda Xie Zheng Zhang Yue Cao Yutong Lin Jianmin Bao Zhuliang Yao Qi Dai Han Hu 194 1,353 0 18 Nov 2021
iBOT: Image BERT Pre-Training with Online Tokenizer Jinghao Zhou Chen Wei Huiyu Wang Wei Shen Cihang Xie Alan Yuille Tao Kong 81 735 0 15 Nov 2021
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 465 7,757 0 11 Nov 2021
BEiT: BERT Pre-Training of Image Transformers Hangbo Bao Li Dong Songhao Piao Furu Wei ViT 274 2,826 0 15 Jun 2021
Self-Supervised Learning with Swin Transformers Zhenda Xie Yutong Lin Zhuliang Yao Zheng Zhang Qi Dai Yue Cao Han Hu ViT 73 183 0 10 May 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 688 6,079 0 29 Apr 2021
An Empirical Study of Training Self-Supervised Vision Transformers Xinlei Chen Saining Xie Kaiming He ViT 154 1,864 0 05 Apr 2021
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows Ze Liu Yutong Lin Yue Cao Han Hu Yixuan Wei Zheng Zhang Stephen Lin B. Guo ViT 450 21,439 0 25 Mar 2021
Zero-Shot Text-to-Image Generation Aditya A. Ramesh Mikhail Pavlov Gabriel Goh Scott Gray Chelsea Voss Alec Radford Mark Chen Ilya Sutskever VLM 415 4,953 0 24 Feb 2021
SEED: Self-supervised Distillation For Visual Representation Zhiyuan Fang Jianfeng Wang Lijuan Wang Lei Zhang Yezhou Yang Zicheng Liu SSL 298 194 0 12 Jan 2021
Training data-efficient image transformers & distillation through attention Hugo Touvron Matthieu Cord Matthijs Douze Francisco Massa Alexandre Sablayrolles Hervé Jégou ViT 387 6,768 0 23 Dec 2020
Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learning Zhenda Xie Yutong Lin Zheng Zhang Yue Cao Stephen Lin Han Hu SSL 109 414 0 19 Nov 2020
CompRess: Self-Supervised Learning by Compressing Representations Soroush Abbasi Koohpayegani Ajinkya Tejankar Hamed Pirsiavash SSL 66 91 0 28 Oct 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 657 41,103 0 22 Oct 2020
Parametric Instance Classification for Unsupervised Visual Feature Learning Yue Cao Zhenda Xie B. Liu Yutong Lin Zheng Zhang Han Hu VLM 67 60 0 25 Jun 2020
Bootstrap your own latent: A new approach to self-supervised Learning Jean-Bastien Grill Florian Strub Florent Altché Corentin Tallec Pierre Harvey Richemond ... M. G. Azar Bilal Piot Koray Kavukcuoglu Rémi Munos Michal Valko SSL 371 6,806 0 13 Jun 2020
A Simple Framework for Contrastive Learning of Visual Representations Ting-Li Chen Simon Kornblith Mohammad Norouzi Geoffrey E. Hinton SSL 369 18,778 0 13 Feb 2020
Momentum Contrast for Unsupervised Visual Representation Learning Kaiming He Haoqi Fan Yuxin Wu Saining Xie Ross B. Girshick SSL 201 12,085 0 13 Nov 2019
Contrastive Representation Distillation Yonglong Tian Dilip Krishnan Phillip Isola 151 1,049 0 23 Oct 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 94,891 0 11 Oct 2018
Unified Perceptual Parsing for Scene Understanding Tete Xiao Yingcheng Liu Bolei Zhou Yuning Jiang Jian Sun OCL VOS 192 1,887 0 26 Jul 2018
Representation Learning with Contrastive Predictive Coding Aaron van den Oord Yazhe Li Oriol Vinyals DRL SSL 320 10,302 0 10 Jul 2018
Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination Zhirong Wu Yuanjun Xiong Stella X. Yu Dahua Lin SSL 173 3,458 0 05 May 2018
Boosting Self-Supervised Learning via Knowledge Transfer M. Noroozi Ananth Vinjimoor Paolo Favaro Hamed Pirsiavash SSL 282 296 0 01 May 2018
Revisiting Oxford and Paris: Large-Scale Image Retrieval Benchmarking Filip Radenovic Ahmet Iscen Giorgos Tolias Yannis Avrithis Ondřej Chum 52 379 0 29 Mar 2018
Unsupervised Representation Learning by Predicting Image Rotations Spyros Gidaris Praveer Singh N. Komodakis OOD SSL DRL 258 3,290 0 21 Mar 2018
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 701 131,652 0 12 Jun 2017
Mask R-CNN Kaiming He Georgia Gkioxari Piotr Dollár Ross B. Girshick ObjD 352 27,195 0 20 Mar 2017
Learning Features by Watching Objects Move Deepak Pathak Ross B. Girshick Piotr Dollár Trevor Darrell Bharath Hariharan SSL VOS OCL 74 525 0 19 Dec 2016
Context Encoders: Feature Learning by Inpainting Deepak Pathak Philipp Krahenbuhl Jeff Donahue Trevor Darrell Alexei A. Efros SSL 67 5,297 0 25 Apr 2016
Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles M. Noroozi Paolo Favaro SSL 171 2,980 0 30 Mar 2016
Colorful Image Colorization Richard Y. Zhang Phillip Isola Alexei A. Efros 133 3,530 0 28 Mar 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.2K 194,020 0 10 Dec 2015
Unsupervised Visual Representation Learning by Context Prediction Carl Doersch Abhinav Gupta Alexei A. Efros DRL SSL 166 2,786 0 19 May 2015
Distilling the Knowledge in a Neural Network Geoffrey E. Hinton Oriol Vinyals J. Dean FedML 362 19,660 0 09 Mar 2015
FitNets: Hints for Thin Deep Nets Adriana Romero Nicolas Ballas Samira Ebrahimi Kahou Antoine Chassang C. Gatta Yoshua Bengio FedML 308 3,887 0 19 Dec 2014
Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networks Alexey Dosovitskiy Philipp Fischer Jost Tobias Springenberg Martin Riedmiller Thomas Brox OOD SSL 87 1,021 0 26 Jun 2014
Do Deep Nets Really Need to be Deep? Lei Jimmy Ba R. Caruana 163 2,119 0 21 Dec 2013