Masked Siamese Networks for Label-Efficient Learning

14 April 2022

Pascal Vincent

Papers citing "Masked Siamese Networks for Label-Efficient Learning"

50 / 226 papers shown

Title
Masked Scene Contrast: A Scalable Framework for Unsupervised 3D Representation Learning Xiaoyang Wu Xin Wen Xihui Liu Hengshuang Zhao 3DPC 112 40 0 24 Mar 2023
The effectiveness of MAE pre-pretraining for billion-scale pretraining Mannat Singh Quentin Duval Kalyan Vasudev Alwala Haoqi Fan Vaibhav Aggarwal ... Piotr Dollár Christoph Feichtenhofer Ross B. Girshick Rohit Girdhar Ishan Misra LRM 113 63 0 23 Mar 2023
ViC-MAE: Self-Supervised Representation Learning from Images and Video with Contrastive Masked Autoencoders J. Hernandez Ruben Villegas Vicente Ordonez SSL 31 2 0 21 Mar 2023
GeoMIM: Towards Better 3D Knowledge Transfer via Masked Image Modeling for Multi-view 3D Understanding Jihao Liu Tai Wang Boxiao Liu Qihang Zhang Yu Liu Hongsheng Li 33 16 0 20 Mar 2023
Stabilizing Transformer Training by Preventing Attention Entropy Collapse Shuangfei Zhai Tatiana Likhomanenko Etai Littwin Dan Busbridge Jason Ramapuram Yizhe Zhang Jiatao Gu J. Susskind AAML 38 64 0 11 Mar 2023
Ignorance is Bliss: Robust Control via Information Gating Manan Tomar Riashat Islam Matthew E. Taylor Sergey Levine Philip Bachman 39 9 0 10 Mar 2023
Towards domain-invariant Self-Supervised Learning with Batch Styles Standardization Marin Scalbert Maria Vakalopoulou Florent Couzinié-Devy 8 2 0 10 Mar 2023
Adapting Contrastive Language-Image Pretrained (CLIP) Models for Out-of-Distribution Detection Nikolas Adaloglou Félix D. P. Michels Tim Kaiser M. Kollmann VLM 29 0 0 10 Mar 2023
Efficient Masked Autoencoders with Self-Consistency Zhaowen Li Yousong Zhu Zhiyang Chen Wei Li Chaoyang Zhao Rui Zhao Ming Tang Jinqiao Wang 47 2 0 28 Feb 2023
Remote Sensing Scene Classification with Masked Image Modeling (MIM) Liya Wang A. Tien 32 3 0 28 Feb 2023
Layer Grafted Pre-training: Bridging Contrastive Learning And Masked Image Modeling For Label-Efficient Representations Ziyu Jiang Yinpeng Chen Mengchen Liu Dongdong Chen Xiyang Dai Lu Yuan Zicheng Liu Zhangyang Wang SSL VLM CLIP 30 16 0 27 Feb 2023
Random Teachers are Good Teachers Felix Sarnthein Gregor Bachmann Sotiris Anagnostidis Thomas Hofmann 19 4 0 23 Feb 2023
Self-supervised network distillation: an effective approach to exploration in sparse reward environments Matej Pecháč M. Chovanec Igor Farkaš 19 3 0 22 Feb 2023
Diffusion Models and Semi-Supervised Learners Benefit Mutually with Few Labels Zebin You Yong Zhong Fan Bao Jiacheng Sun Chongxuan Li Jun Zhu DiffM VLM 201 36 0 21 Feb 2023
Object-centric Learning with Cyclic Walks between Parts and Whole Ziyu Wang Mike Zheng Shou Mengmi Zhang OCL 30 9 0 16 Feb 2023
Self-supervised pseudo-colorizing of masked cells Royden Wagner Carlos Fernandez Lopez Christoph Stiller 15 0 0 12 Feb 2023
Evaluating Self-Supervised Learning via Risk Decomposition Yann Dubois Tatsunori Hashimoto Percy Liang 14 9 0 06 Feb 2023
Energy-Inspired Self-Supervised Pretraining for Vision Models Ze Wang Jiang Wang Zicheng Liu Qiang Qiu 21 8 0 02 Feb 2023
Aerial Image Object Detection With Vision Transformer Detector (ViTDet) Liya Wang A. Tien 39 7 0 28 Jan 2023
Cross-Architectural Positive Pairs improve the effectiveness of Self-Supervised Learning P. Singh Jacopo Cirrone SSL 40 0 0 27 Jan 2023
A Simple Recipe for Competitive Low-compute Self supervised Vision Models Quentin Duval Ishan Misra Nicolas Ballas 29 9 0 23 Jan 2023
Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture Mahmoud Assran Quentin Duval Ishan Misra Piotr Bojanowski Pascal Vincent Michael G. Rabbat Yann LeCun Nicolas Ballas SSL AI4TS MDE 27 318 0 19 Jan 2023
Learning Customized Visual Models with Retrieval-Augmented Knowledge Haotian Liu Kilho Son Jianwei Yang Ce Liu Jianfeng Gao Yong Jae Lee Chunyuan Li VLM 38 53 0 17 Jan 2023
RILS: Masked Visual Reconstruction in Language Semantic Space Shusheng Yang Yixiao Ge Kun Yi Dian Li Ying Shan Xiaohu Qie Xinggang Wang CLIP 40 11 0 17 Jan 2023
RxRx1: A Dataset for Evaluating Experimental Batch Correction Methods Maciej Sypetkowski Morteza Rezanejad Saber Saberian Oren Z. Kraus John Urbanik ... Mason L. Victors J. Yosinski A. R. Sereshkeh I. Haque Berton A. Earnshaw 31 37 0 13 Jan 2023
Swin MAE: Masked Autoencoders for Small Datasets Zián Xu Yin Dai Fayu Liu Weibin Chen Yue Liu Li-Li Shi Sheng Liu Yuhang Zhou SyDa MedIm ViT 28 28 0 28 Dec 2022
Attentive Mask CLIP Yifan Yang Weiquan Huang Yixuan Wei Houwen Peng Xinyang Jiang ... Fangyun Wei Yin Wang Han Hu Lili Qiu Yuqing Yang CLIP VLM 34 26 0 16 Dec 2022
Improving self-supervised representation learning via sequential adversarial masking Dylan Sam Min Bai Tristan McKinney Li Erran Li SSL 19 0 0 16 Dec 2022
Efficient Self-supervised Learning with Contextualized Target Representations for Vision, Speech and Language Alexei Baevski Arun Babu Wei-Ning Hsu Michael Auli VLM SSL 32 91 0 14 Dec 2022
Co-training $2^L$ Submodels for Visual Recognition Hugo Touvron Matthieu Cord Maxime Oquab Piotr Bojanowski Jakob Verbeek Hervé Jégou VLM 27 9 0 09 Dec 2022
Implicit variance regularization in non-contrastive SSL Manu Srinath Halvagal Axel Laborieux Friedemann Zenke 34 8 0 09 Dec 2022
Location-Aware Self-Supervised Transformers for Semantic Segmentation Mathilde Caron N. Houlsby Cordelia Schmid ViT 16 10 0 05 Dec 2022
Spatio-Temporal Crop Aggregation for Video Representation Learning Sepehr Sameni Simon Jenni Paolo Favaro 15 3 0 30 Nov 2022
Self-Supervised Learning based on Heat Equation Yinpeng Chen Xiyang Dai Dongdong Chen Mengchen Liu Lu Yuan Zicheng Liu Youzuo Lin 29 4 0 23 Nov 2022
Fast-iTPN: Integrally Pre-Trained Transformer Pyramid Network with Token Migration Yunjie Tian Lingxi Xie Jihao Qiu Jianbin Jiao Yaowei Wang Qi Tian Qixiang Ye ViT 29 6 0 23 Nov 2022
Parametric Classification for Generalized Category Discovery: A Baseline Study Xin Wen Bingchen Zhao Xiaojuan Qi 30 67 0 21 Nov 2022
Contrastive Masked Autoencoders for Self-Supervised Video Hashing Yuting Wang Jinpeng Wang B. Chen Ziyun Zeng Shutao Xia 29 20 0 21 Nov 2022
CroCo v2: Improved Cross-view Completion Pre-training for Stereo Matching and Optical Flow Philippe Weinzaepfel Thomas Lucas Vincent Leroy Yohann Cabon Vaibhav Arora Romain Brégier G. Csurka L. Antsfeld Boris Chidlovskii Jérôme Revaud ViT 18 80 0 18 Nov 2022
Weighted Ensemble Self-Supervised Learning Yangjun Ruan Saurabh Singh Warren Morningstar Alexander A. Alemi Sergey Ioffe Ian S. Fischer Joshua V. Dillon FedML 22 15 0 18 Nov 2022
Towards All-in-one Pre-training via Maximizing Multi-modal Mutual Information Weijie Su Xizhou Zhu Chenxin Tao Lewei Lu Bin Li Gao Huang Yu Qiao Xiaogang Wang Jie Zhou Jifeng Dai 34 41 0 17 Nov 2022
CAE v2: Context Autoencoder with CLIP Target Xinyu Zhang Jiahui Chen Junkun Yuan Qiang Chen Jian Wang ... Jimin Pi Kun Yao Junyu Han Errui Ding Jingdong Wang VLM CLIP 42 24 0 17 Nov 2022
MAGE: MAsked Generative Encoder to Unify Representation Learning and Image Synthesis Tianhong Li Huiwen Chang Shlok Kumar Mishra Han Zhang Dina Katabi Dilip Krishnan 32 152 0 16 Nov 2022
Masked Contrastive Representation Learning Yuan Yao Nandakishor Desai M. Palaniswami SSL 14 8 0 11 Nov 2022
A simple, efficient and scalable contrastive masked autoencoder for learning visual representations Shlok Kumar Mishra Joshua Robinson Huiwen Chang David Jacobs Aaron Sarna Aaron Maschinot Dilip Krishnan DiffM 43 30 0 30 Oct 2022
Masked Modeling Duo: Learning Representations by Encouraging Both Networks to Model the Input Daisuke Niizumi Daiki Takeuchi Yasunori Ohishi Noboru Harada K. Kashino SSL 32 30 0 26 Oct 2022
Neural Eigenfunctions Are Structured Representation Learners Zhijie Deng Jiaxin Shi Hao Zhang Peng Cui Cewu Lu Jun Zhu 45 14 0 23 Oct 2022
MixMask: Revisiting Masking Strategy for Siamese ConvNets Kirill Vishniakov Eric P. Xing Zhiqiang Shen 16 0 0 20 Oct 2022
CroCo: Self-Supervised Pre-training for 3D Vision Tasks by Cross-View Completion Philippe Weinzaepfel Vincent Leroy Thomas Lucas Romain Brégier Yohann Cabon Vaibhav Arora L. Antsfeld Boris Chidlovskii G. Csurka Jérôme Revaud SSL 31 64 0 19 Oct 2022
A Unified View of Masked Image Modeling Zhiliang Peng Li Dong Hangbo Bao QiXiang Ye Furu Wei VLM 52 35 0 19 Oct 2022
Non-Contrastive Learning Meets Language-Image Pre-Training Jinghao Zhou Li Dong Zhe Gan Lijuan Wang Furu Wei VLM CLIP 17 26 0 17 Oct 2022