Masked Autoencoders Enable Efficient Knowledge Distillers

25 August 2022

Cihang Xie

Papers citing "Masked Autoencoders Enable Efficient Knowledge Distillers"

27 / 27 papers shown

Title
EmbodiedMAE: A Unified 3D Multi-Modal Representation for Robot Manipulation Zibin Dong Fei Ni Yifu Yuan Yinchuan Li Jianye Hao 26 0 0 15 May 2025
Crossmodal Knowledge Distillation with WordNet-Relaxed Text Embeddings for Robust Image Classification Chenqi Guo Mengshuo Rong Qianli Feng Rongfan Feng Yinglong Ma VLM 68 0 0 31 Mar 2025
UNIP: Rethinking Pre-trained Attention Patterns for Infrared Semantic Segmentation Tao Zhang Jinyong Wen Zhen Chen Kun Ding S. Xiang Chunhong Pan 72 1 0 04 Feb 2025
Multi-Level Feature Distillation of Joint Teachers Trained on Distinct Image Datasets Adrian Iordache B. Alexe Radu Tudor Ionescu 31 1 0 29 Oct 2024
Masked Image Modeling: A Survey Vlad Hondru Florinel-Alin Croitoru Shervin Minaee Radu Tudor Ionescu N. Sebe 69 6 0 13 Aug 2024
Unveiling Incomplete Modality Brain Tumor Segmentation: Leveraging Masked Predicted Auto-Encoder and Divergence Learning Zhongao Sun Jiameng Li Yuhan Wang Jiarong Cheng Qing Zhou Chun Li MedIm 28 0 0 12 Jun 2024
Why does Knowledge Distillation Work? Rethink its Attention and Fidelity Mechanism Chenqi Guo Shiwei Zhong Xiaofeng Liu Qianli Feng Yinglong Ma 24 1 0 30 Apr 2024
Representing Part-Whole Hierarchies in Foundation Models by Learning Localizability, Composability, and Decomposability from Anatomy via Self-Supervision M. Taher Michael B. Gotway Jianming Liang MedIm 31 5 0 24 Apr 2024
Weight Copy and Low-Rank Adaptation for Few-Shot Distillation of Vision Transformers Diana-Nicoleta Grigore Mariana-Iuliana Georgescu J. A. Justo T. Johansen Andreea-Iuliana Ionescu Radu Tudor Ionescu 36 0 0 14 Apr 2024
SyncMask: Synchronized Attentional Masking for Fashion-centric Vision-Language Pretraining Chull Hwan Song Taebaek Hwang Jooyoung Yoon Shunghyun Choi Yeong Hyeon Gu 23 4 0 01 Apr 2024
Once for Both: Single Stage of Importance and Sparsity Search for Vision Transformer Compression Hancheng Ye Chong Yu Peng Ye Renqiu Xia Yansong Tang Jiwen Lu Tao Chen Bo-Wen Zhang 53 3 0 23 Mar 2024
SELECTOR: Heterogeneous graph network with convolutional masked autoencoder for multimodal robust prediction of cancer survival Liangrui Pan Yijun Peng Yan Li Xiang Wang Wenjuan Liu Liwen Xu Qingchun Liang Shaoliang Peng 40 3 0 14 Mar 2024
Task-customized Masked AutoEncoder via Mixture of Cluster-conditional Experts Zhili Liu Kai Chen Jianhua Han Lanqing Hong Hang Xu Zhenguo Li James T. Kwok MoE 111 24 0 08 Feb 2024
Masked Modeling for Self-supervised Representation Learning on Vision and Beyond Siyuan Li Luyuan Zhang Zedong Wang Di Wu Lirong Wu ... Jun-Xiong Xia Cheng Tan Yang Liu Baigui Sun Stan Z. Li SSL 36 14 0 31 Dec 2023
EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment Anything Yunyang Xiong Bala Varadarajan Lemeng Wu Xiaoyu Xiang Fanyi Xiao ... Dilin Wang Fei Sun Forrest N. Iandola Raghuraman Krishnamoorthi Vikas Chandra VLM 40 139 0 01 Dec 2023
Asymmetric Masked Distillation for Pre-Training Small Foundation Models Zhiyu Zhao Bingkun Huang Sen Xing Gangshan Wu Yu Qiao Limin Wang 39 5 0 06 Nov 2023
CM-MaskSD: Cross-Modality Masked Self-Distillation for Referring Image Segmentation Wenxuan Wang Jing Liu Xingjian He Yisi Zhang Cheng Chen Jiachen Shen Yan Zhang Jiangyun Li 19 11 0 19 May 2023
Mimic before Reconstruct: Enhancing Masked Autoencoders with Feature Mimicking Peng Gao Renrui Zhang Rongyao Fang Ziyi Lin Hongyang Li Hongsheng Li Qiao Yu 19 18 0 09 Mar 2023
OmniForce: On Human-Centered, Large Model Empowered and Cloud-Edge Collaborative AutoML System Chao Xue Wei Liu Shunxing Xie Zhenfang Wang Jiaxing Li ... Shi-Yong Chen Yibing Zhan Jing Zhang Chaoyue Wang Dacheng Tao 43 2 0 01 Mar 2023
Generic-to-Specific Distillation of Masked Autoencoders Wei Huang Zhiliang Peng Li Dong Furu Wei Jianbin Jiao QiXiang Ye 32 22 0 28 Feb 2023
Remote Sensing Scene Classification with Masked Image Modeling (MIM) Liya Wang A. Tien 35 3 0 28 Feb 2023
MOMA:Distill from Self-Supervised Teachers Yuan Yao Nandakishor Desai M. Palaniswami 32 2 0 04 Feb 2023
Aerial Image Object Detection With Vision Transformer Detector (ViTDet) Liya Wang A. Tien 44 7 0 28 Jan 2023
Stare at What You See: Masked Image Modeling without Reconstruction Hongwei Xue Peng Gao Hongyang Li Yu Qiao Hao Sun Houqiang Li Jiebo Luo 25 31 0 16 Nov 2022
It Takes Two: Masked Appearance-Motion Modeling for Self-supervised Video Transformer Pre-training Yuxin Song Min Yang Wenhao Wu Dongliang He Fu Li Jingdong Wang ViT 95 8 0 11 Oct 2022
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 305 7,443 0 11 Nov 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 317 5,785 0 29 Apr 2021