SdAE: Self-distillated Masked Autoencoder

31 July 2022

ArXiv (abs)PDF HTML Github (60★)

Papers citing "SdAE: Self-distillated Masked Autoencoder"

46 / 46 papers shown

Title
C3R: Channel Conditioned Cell Representations for unified evaluation in microscopy imaging Umar Marikkar Syed Sameed Husain Muhammad Awais Sara Atito 49 0 0 24 May 2025
Uni4D: A Unified Self-Supervised Learning Framework for Point Cloud Videos Zhi Zuo Chenyi Zhuang Zhiqiang Shen Pan Gao Jie Qin Nicu Sebe 3DPC 123 0 0 07 Apr 2025
Identifying Surgical Instruments in Pedagogical Cataract Surgery Videos through an Optimized Aggregation Network Sanya Sinha Michal Balazia Francois Bremond 107 0 0 05 Jan 2025
KDC-MAE: Knowledge Distilled Contrastive Mask Auto-Encoder Maheswar Bora Saurabh Atreya Aritra Mukherjee Abhijit Das 140 0 0 19 Nov 2024
From Prototypes to General Distributions: An Efficient Curriculum for Masked Image Modeling Jinhong Lin Cheng-En Wu Huanran Li Jifan Zhang Yu Hen Hu Pedro Morgado 121 0 0 16 Nov 2024
Breaking Modality Gap in RGBT Tracking: Coupled Knowledge Distillation Andong Lu Jiacong Zhao Chenglong Li Yun Xiao Bin Luo 98 4 0 15 Oct 2024
Masked Image Modeling: A Survey Vlad Hondru Florinel-Alin Croitoru Shervin Minaee Radu Tudor Ionescu N. Sebe 209 8 0 13 Aug 2024
POA: Pre-training Once for Models of All Sizes Yingying Zhang Xin Guo Jiangwei Lao Lei Yu Lixiang Ru Jian Wang Guo Ye Huimei He Jingdong Chen Ming Yang 171 1 0 02 Aug 2024
Pseudo Labelling for Enhanced Masked Autoencoders S. Nandam Sara Atito Zhenhua Feng Josef Kittler Muhammad Awais 106 1 0 25 Jun 2024
SemanticMIM: Marring Masked Image Modeling with Semantics Compression for General Visual Representation Yike Yuan Huanzhang Dou Fengjun Guo Xi Li 104 2 0 15 Jun 2024
DailyMAE: Towards Pretraining Masked Autoencoders in One Day Jiantao Wu Shentong Mo Sara Atito Zhenhua Feng Josef Kittler Muhammad Awais 84 3 0 31 Mar 2024
YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information Chien-Yao Wang I-Hau Yeh Hongpeng Liao 111 1,411 0 21 Feb 2024
Masked Modeling for Self-supervised Representation Learning on Vision and Beyond Siyuan Li Luyuan Zhang Zedong Wang Di Wu Lirong Wu ... Jun Xia Cheng Tan Yang Liu Baigui Sun Stan Z. Li SSL 118 15 0 31 Dec 2023
Morphing Tokens Draw Strong Masked Image Models Taekyung Kim Byeongho Heo Dongyoon Han 194 3 0 30 Dec 2023
Learning to Embed Time Series Patches Independently Seunghan Lee Taeyoung Park Kibok Lee SSL AI4TS 95 32 0 27 Dec 2023
Cascade-Zero123: One Image to Highly Consistent 3D with Self-Prompted Nearby Views Yabo Chen Jiemin Fang Yuyang Huang Taoran Yi Xiaopeng Zhang Lingxi Xie Xinggang Wang Wenrui Dai Hongkai Xiong Qi Tian DiffM 91 21 0 07 Dec 2023
Rejuvenating image-GPT as Strong Visual Representation Learners Sucheng Ren Zeyu Wang Hongru Zhu Junfei Xiao Alan Yuille Cihang Xie VLM 123 8 0 04 Dec 2023
Betrayed by Attention: A Simple yet Effective Approach for Self-supervised Video Object Segmentation Shuangrui Ding Rui Qian Haohang Xu Dahua Lin Hongkai Xiong VOS 102 5 0 29 Nov 2023
Towards Transferable Multi-modal Perception Representation Learning for Autonomy: NeRF-Supervised Masked AutoEncoder Xiaohao Xu 138 0 0 23 Nov 2023
Asymmetric Masked Distillation for Pre-Training Small Foundation Models Zhiyu Zhao Bingkun Huang Sen Xing Gangshan Wu Yu Qiao Limin Wang 89 5 0 06 Nov 2023
Learning with Unmasked Tokens Drives Stronger Vision Learners Taekyung Kim Sanghyuk Chun Byeongho Heo Dongyoon Han SSL 100 2 0 20 Oct 2023
Masked Image Residual Learning for Scaling Deeper Vision Transformers Guoxi Huang Hongtao Fu A. Bors 122 7 0 25 Sep 2023
Beyond Image Borders: Learning Feature Extrapolation for Unbounded Image Composition Xiaoyu Liu Ming-Yu Liu Junyi Li Shuai Liu Xiaotao Wang Lei Lei Wangmeng Zuo 75 2 0 21 Sep 2023
RevColV2: Exploring Disentangled Representations in Masked Image Modeling Qi Han Yuxuan Cai Xiangyu Zhang 123 8 0 02 Sep 2023
Contrastive Feature Masking Open-Vocabulary Vision Transformer Dahun Kim A. Angelova Weicheng Kuo ObjD VLM 120 27 0 02 Sep 2023
MOFO: MOtion FOcused Self-Supervision for Video Understanding Mona Ahmadian Frank Guerin Andrew Gilbert 83 2 0 23 Aug 2023
Hybrid Distillation: Connecting Masked Autoencoders with Contrastive Learners Bowen Shi Xiaopeng Zhang Yaoming Wang Jin Li Wenrui Dai Junni Zou H. Xiong Qi Tian 101 4 0 28 Jun 2023
Self-Distilled Masked Auto-Encoders are Efficient Video Anomaly Detectors Nicolae-Cătălin Ristea Florinel-Alin Croitoru Radu Tudor Ionescu Marius Popescu Fahad Shahbaz Khan M. Shah ViT 121 23 0 21 Jun 2023
Know Your Self-supervised Learning: A Survey on Image-based Generative and Discriminative Training Utku Ozbulak Hyun Jung Lee Beril Boga Esla Timothy Anzaku Ho-min Park Arnout Van Messem W. D. Neve J. Vankerschaver DiffM 112 38 0 23 May 2023
Graph Masked Autoencoder for Sequential Recommendation Yaowen Ye Lianghao Xia Chao Huang 106 45 0 08 May 2023
Mask and Restore: Blind Backdoor Defense at Test Time with Masked Autoencoder Tao Sun Lu Pang Chao Chen Haibin Ling AAML 75 9 0 27 Mar 2023
Centroid-centered Modeling for Efficient Vision Transformer Pre-training Xin Yan Zuchao Li Lefei Zhang Bo Du Dacheng Tao VLM 73 0 0 08 Mar 2023
Remote Sensing Scene Classification with Masked Image Modeling (MIM) Liya Wang A. Tien 98 3 0 28 Feb 2023
Anatomical Invariance Modeling and Semantic Alignment for Self-supervised Learning in 3D Medical Image Analysis Yankai Jiang Ming Sun Heng Guo Xiaoyu Bai K. Yan Le Lu Minfeng Xu MedIm 130 22 0 11 Feb 2023
Beyond Pretrained Features: Noisy Image Modeling Provides Adversarial Defense Zunzhi You Daochang Liu Bohyung Han Chang Xu AAML VLM 119 5 0 02 Feb 2023
Aerial Image Object Detection With Vision Transformer Detector (ViTDet) Liya Wang A. Tien 143 9 0 28 Jan 2023
RILS: Masked Visual Reconstruction in Language Semantic Space Shusheng Yang Yixiao Ge Kun Yi Dian Li Ying Shan Xiaohu Qie Xinggang Wang CLIP 95 11 0 17 Jan 2023
A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trends Jie Gui Tuo Chen Jing Zhang Qiong Cao Zhe Sun Haoran Luo Dacheng Tao 235 161 0 13 Jan 2023
TinyMIM: An Empirical Study of Distilling MIM Pre-trained Models Sucheng Ren Fangyun Wei Zheng Zhang Han Hu 146 43 0 03 Jan 2023
Disjoint Masking with Joint Distillation for Efficient Masked Image Modeling Xin Ma Chang-Shu Liu Chunyu Xie Long Ye Yafeng Deng Xiang Ji 145 10 0 31 Dec 2022
Masked autoencoders are effective solution to transformer data-hungry Jia-ju Mao Honggu Zhou Xuesong Yin Binling Nie MedIm 128 7 0 12 Dec 2022
Masked Video Distillation: Rethinking Masked Feature Modeling for Self-supervised Video Representation Learning Rui Wang Dongdong Chen Zuxuan Wu Yinpeng Chen Xiyang Dai Mengchen Liu Lu Yuan Yu-Gang Jiang VGen 128 94 0 08 Dec 2022
Fast-iTPN: Integrally Pre-Trained Transformer Pyramid Network with Token Migration Yunjie Tian Lingxi Xie Jihao Qiu Jianbin Jiao Yaowei Wang Qi Tian Qixiang Ye ViT 98 7 0 23 Nov 2022
CAE v2: Context Autoencoder with CLIP Target Xinyu Zhang Jiahui Chen Junkun Yuan Qiang Chen Jian Wang ... Jimin Pi Kun Yao Junyu Han Errui Ding Jingdong Wang VLM CLIP 109 24 0 17 Nov 2022
A Unified View of Masked Image Modeling Zhiliang Peng Li Dong Hangbo Bao QiXiang Ye Furu Wei VLM 133 38 0 19 Oct 2022
Unsupervised Learning of Visual Features by Contrasting Cluster Assignments Mathilde Caron Ishan Misra Julien Mairal Priya Goyal Piotr Bojanowski Armand Joulin OCL SSL 355 4,115 0 17 Jun 2020