v1v2 (latest)

Beyond [cls]: Exploring the true potential of Masked Image Modeling representations

4 December 2024

Marcin Przewiȩźlikowski

Papers citing "Beyond [cls]: Exploring the true potential of Masked Image Modeling representations"

50 / 57 papers shown

Title
Attention, Please! Revisiting Attentive Probing for Masked Image Modeling Bill Psomas Dionysis Christopoulos Eirini Baltzi Ioannis Kakogeorgiou Tilemachos Aravanis N. Komodakis Konstantinos Karantzalos Yannis Avrithis Giorgos Tolias 56 0 0 11 Jun 2025
SemanticMIM: Marring Masked Image Modeling with Semantics Compression for General Visual Representation Yike Yuan Huanzhang Dou Fengjun Guo Xi Li 99 2 0 15 Jun 2024
AdaGlimpse: Active Visual Exploration with Arbitrary Glimpse Position and Scale Adam Pardyl Michal Wronka Maciej Wolczyk Kamil Adamczewski Tomasz Trzciñski Bartosz Zieliñski 85 2 0 04 Apr 2024
Learning by Reconstruction Produces Uninformative Features For Perception Randall Balestriero Yann LeCun 70 23 0 17 Feb 2024
Revisiting Feature Prediction for Learning Visual Representations from Video Adrien Bardes Q. Garrido Jean Ponce Xinlei Chen Michael G. Rabbat Yann LeCun Mahmoud Assran Nicolas Ballas MDE VLM 155 87 0 15 Feb 2024
Is ImageNet worth 1 video? Learning strong image encoders from 1 long unlabelled video Shashanka Venkataramanan Mamshad Nayeem Rizve João Carreira Yuki M. Asano Yannis Avrithis SSL 66 20 0 12 Oct 2023
Keep It SimPool: Who Said Supervised Transformers Suffer from Attention Deficit? Bill Psomas Ioannis Kakogeorgiou Konstantinos Karantzalos Yannis Avrithis ViT 71 10 0 13 Sep 2023
Augmentation-aware Self-supervised Learning with Conditioned Projector Marcin Przewike'zlikowski Mateusz Pyla Bartosz Zieliñski Bartlomiej Twardowski Jacek Tabor Marek Śmieja SSL 93 3 0 31 May 2023
What Do Self-Supervised Vision Transformers Learn? Namuk Park Wonjae Kim Byeongho Heo Taekyung Kim Sangdoo Yun SSL 176 80 1 01 May 2023
A Cookbook of Self-Supervised Learning Randall Balestriero Mark Ibrahim Vlad Sobal Ari S. Morcos Shashank Shekhar ... Pierre Fernandez Amir Bar Hamed Pirsiavash Yann LeCun Micah Goldblum SyDa FedML SSL 154 284 0 24 Apr 2023
DINOv2: Learning Robust Visual Features without Supervision Maxime Oquab Timothée Darcet Théo Moutakanni Huy Q. Vo Marc Szafraniec ... Hervé Jégou Julien Mairal Patrick Labatut Armand Joulin Piotr Bojanowski VLM CLIP SSL 448 3,529 0 14 Apr 2023
Active Visual Exploration Based on Attention-Map Entropy Adam Pardyl Grzegorz Rype'sć Grzegorz Kurzejamski Bartosz Zieliñski Tomasz Trzciñski 93 6 0 11 Mar 2023
Amortised Invariance Learning for Contrastive Self-Supervision Ruchika Chavhan Henry Gouk Jan Stuehmer Calum Heggan Mehrdad Yaghoobi Timothy M. Hospedales SSL 88 11 0 24 Feb 2023
Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture Mahmoud Assran Quentin Duval Ishan Misra Piotr Bojanowski Pascal Vincent Michael G. Rabbat Yann LeCun Nicolas Ballas SSL AI4TS MDE 143 361 0 19 Jan 2023
How Mask Matters: Towards Theoretical Understandings of Masked Autoencoders Qi Zhang Yifei Wang Yisen Wang 79 76 0 15 Oct 2022
The Hidden Uniform Cluster Prior in Self-Supervised Learning Mahmoud Assran Randall Balestriero Quentin Duval Florian Bordes Ishan Misra Piotr Bojanowski Pascal Vincent Michael G. Rabbat Nicolas Ballas SSL 96 50 0 13 Oct 2022
VICRegL: Self-Supervised Learning of Local Visual Features Adrien Bardes Jean Ponce Yann LeCun SSL 99 126 0 04 Oct 2022
Exploring Target Representations for Masked Autoencoders Xingbin Liu Jinghao Zhou Tao Kong Xianming Lin Rongrong Ji 195 52 0 08 Sep 2022
BEiT v2: Masked Image Modeling with Vector-Quantized Visual Tokenizers Zhiliang Peng Li Dong Hangbo Bao QiXiang Ye Furu Wei 71 322 0 12 Aug 2022
Understanding Masked Image Modeling via Learning Occlusion Invariant Feature Xiangwen Kong Xiangyu Zhang SSL 75 54 0 08 Aug 2022
Guillotine Regularization: Why removing layers is needed to improve generalization in Self-Supervised Learning Florian Bordes Randall Balestriero Q. Garrido Adrien Bardes Pascal Vincent 115 24 0 27 Jun 2022
Masked Siamese Networks for Label-Efficient Learning Mahmoud Assran Mathilde Caron Ishan Misra Piotr Bojanowski Florian Bordes Pascal Vincent Armand Joulin Michael G. Rabbat Nicolas Ballas SSL 131 325 0 14 Apr 2022
Context Autoencoder for Self-Supervised Representation Learning Xiaokang Chen Mingyu Ding Xiaodi Wang Ying Xin Shentong Mo Yunhao Wang Shumin Han Ping Luo Gang Zeng Jingdong Wang SSL 193 400 0 07 Feb 2022
Masked Feature Prediction for Self-Supervised Visual Pre-Training Chen Wei Haoqi Fan Saining Xie Chaoxia Wu Alan Yuille Christoph Feichtenhofer ViT 184 674 0 16 Dec 2021
SimMIM: A Simple Framework for Masked Image Modeling Zhenda Xie Zheng Zhang Yue Cao Yutong Lin Jianmin Bao Zhuliang Yao Qi Dai Han Hu 246 1,371 0 18 Nov 2021
Improving Transferability of Representations via Augmentation-Aware Self-Supervision Hankook Lee Kibok Lee Kimin Lee Honglak Lee Jinwoo Shin SSL 85 55 0 18 Nov 2021
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 507 7,865 0 11 Nov 2021
BEiT: BERT Pre-Training of Image Transformers Hangbo Bao Li Dong Songhao Piao Furu Wei ViT 328 2,853 0 15 Jun 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 821 6,149 0 29 Apr 2021
Condenser: a Pre-training Architecture for Dense Retrieval Luyu Gao Jamie Callan AI4CE 59 267 0 16 Apr 2021
SiT: Self-supervised vIsion Transformer Sara Atito Ali Ahmed Muhammad Awais J. Kittler ViT 110 139 0 08 Apr 2021
An Empirical Study of Training Self-Supervised Vision Transformers Xinlei Chen Saining Xie Kaiming He ViT 179 1,874 0 05 Apr 2021
Barlow Twins: Self-Supervised Learning via Redundancy Reduction Jure Zbontar Li Jing Ishan Misra Yann LeCun Stéphane Deny SSL 360 2,372 0 04 Mar 2021
Exploring Simple Siamese Representation Learning Xinlei Chen Kaiming He SSL 266 4,081 0 20 Nov 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 738 41,796 0 22 Oct 2020
What Should Not Be Contrastive in Contrastive Learning Tete Xiao Xiaolong Wang Alexei A. Efros Trevor Darrell SSL DRL 111 303 0 13 Aug 2020
Unsupervised Learning of Visual Features by Contrasting Cluster Assignments Mathilde Caron Ishan Misra Julien Mairal Priya Goyal Piotr Bojanowski Armand Joulin OCL SSL 334 4,109 0 17 Jun 2020
Bootstrap your own latent: A new approach to self-supervised Learning Jean-Bastien Grill Florian Strub Florent Altché Corentin Tallec Pierre Harvey Richemond ... M. G. Azar Bilal Piot Koray Kavukcuoglu Rémi Munos Michal Valko SSL 459 6,864 0 13 Jun 2020
What Makes for Good Views for Contrastive Learning? Yonglong Tian Chen Sun Ben Poole Dilip Krishnan Cordelia Schmid Phillip Isola SSL 120 1,340 0 20 May 2020
Improved Baselines with Momentum Contrastive Learning Xinlei Chen Haoqi Fan Ross B. Girshick Kaiming He SSL 559 3,450 0 09 Mar 2020
A Simple Framework for Contrastive Learning of Visual Representations Ting-Li Chen Simon Kornblith Mohammad Norouzi Geoffrey E. Hinton SSL 422 18,968 0 13 Feb 2020
Evaluating Weakly Supervised Object Localization Methods Right Junsuk Choe Seong Joon Oh Seungho Lee Sanghyuk Chun Zeynep Akata Hyunjung Shim WSOL 403 189 0 21 Jan 2020
PyTorch: An Imperative Style, High-Performance Deep Learning Library Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury ... Sasank Chilamkurthy Benoit Steiner Lu Fang Junjie Bai Soumith Chintala ODL 632 42,770 0 03 Dec 2019
Momentum Contrast for Unsupervised Visual Representation Learning Kaiming He Haoqi Fan Yuxin Wu Saining Xie Ross B. Girshick SSL 261 12,164 0 13 Nov 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.9K 95,531 0 11 Oct 2018
Representation Learning with Contrastive Predictive Coding Aaron van den Oord Yazhe Li Oriol Vinyals DRL SSL 360 10,385 0 10 Jul 2018
Unsupervised Representation Learning by Predicting Image Rotations Spyros Gidaris Praveer Singh N. Komodakis OOD SSL DRL 275 3,304 0 21 Mar 2018
Attention-based Deep Multiple Instance Learning Maximilian Ilse Jakub M. Tomczak Max Welling 211 1,841 0 13 Feb 2018
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 939 133,201 0 12 Jun 2017
Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles M. Noroozi Paolo Favaro SSL 222 19 0 30 Mar 2016