A Closer Look at Self-Supervised Lightweight Vision Transformers

28 May 2022

Papers citing "A Closer Look at Self-Supervised Lightweight Vision Transformers"

32 / 32 papers shown

Title
AlignDiff: Learning Physically-Grounded Camera Alignment via Diffusion Liuyue Xie Jiancong Guo Ozan Cakmakci Andre Araujo László A. Jeni Zhiheng Jia DiffM 60 0 0 27 Mar 2025
Scale-Aware Pre-Training for Human-Centric Visual Perception: Enabling Lightweight and Generalizable Models Xuanhan Wang Huimin Deng Lianli Gao Jingkuan Song VLM 59 0 0 11 Mar 2025
Task-Specific Knowledge Distillation from the Vision Foundation Model for Enhanced Medical Image Segmentation Pengchen Liang Haishan Huang Bin Pu Jianguo Chen Xiang Hua Jing Zhang Weibo Ma Z. Chen Yiwei Li Qing Chang 43 0 0 10 Mar 2025
UNIP: Rethinking Pre-trained Attention Patterns for Infrared Semantic Segmentation Tao Zhang Jinyong Wen Zhen Chen Kun Ding S. Xiang Chunhong Pan 72 1 0 04 Feb 2025
DCT-HistoTransformer: Efficient Lightweight Vision Transformer with DCT Integration for histopathological image analysis Mahtab Ranjbar Mehdi Mohebbi Mahdi Cherakhloo Bijan Vosoughi. Vahdat MedIm 21 0 0 24 Oct 2024
BTMuda: A Bi-level Multi-source unsupervised domain adaptation framework for breast cancer diagnosis Yuxiang Yang Xinyi Zeng Pinxian Zeng Binyu Yan Xi Wu Jiliu Zhou Yan Wang OOD 23 1 0 30 Aug 2024
MLIP: Efficient Multi-Perspective Language-Image Pretraining with Exhaustive Data Utilization Yu Zhang Qi Zhang Zixuan Gong Yiwei Shi Yepeng Liu ... Ke Liu Kun Yi Wei Fan Liang Hu Changwei Wang CLIP VLM 51 3 0 03 Jun 2024
HMANet: Hybrid Multi-Axis Aggregation Network for Image Super-Resolution S. Chu Zhi-chao Dou Jeng-Shyang Pan Shaowei Weng Junbao Li ViT 31 4 0 08 May 2024
An Experimental Study on Exploring Strong Lightweight Vision Transformers via Masked Image Modeling Pre-Training Jin Gao Shubo Lin Shaoru Wang Yutong Kou Zeming Li Liang Li Congxuan Zhang Xiaoqin Zhang Yizheng Wang Weiming Hu 39 1 0 18 Apr 2024
DailyMAE: Towards Pretraining Masked Autoencoders in One Day Jiantao Wu Shentong Mo Sara Atito Zhenhua Feng Josef Kittler Muhammad Awais 35 3 0 31 Mar 2024
EfficientMorph: Parameter-Efficient Transformer-Based Architecture for 3D Image Registration Abu Zahid Bin Aziz Mokshagna Sai Teja Karanam Tushar Kataria Shireen Elhabian ViT MedIm 29 1 0 16 Mar 2024
Masked Modeling for Self-supervised Representation Learning on Vision and Beyond Siyuan Li Luyuan Zhang Zedong Wang Di Wu Lirong Wu ... Jun-Xiong Xia Cheng Tan Yang Liu Baigui Sun Stan Z. Li SSL 33 14 0 31 Dec 2023
ZoomTrack: Target-aware Non-uniform Resizing for Efficient Visual Tracking Yutong Kou Jin Gao Bing Li Gang Wang Weiming Hu Yizheng Wang Liang Li ViT 36 19 0 16 Oct 2023
Masked Image Residual Learning for Scaling Deeper Vision Transformers Guoxi Huang Hongtao Fu A. Bors 26 7 0 25 Sep 2023
Masked Transformer for Electrocardiogram Classification Ya Zhou Xiaolin Diao Yanni Huo Yang Liu Xiaohan Fan Wei-Ye Zhao MedIm 14 2 0 31 Aug 2023
CLIP Brings Better Features to Visual Aesthetics Learners Liwu Xu Jinjin Xu Yuzhe Yang Yi-Jie Huang Yanchun Xie Yaqian Li VLM 25 3 0 28 Jul 2023
Hybrid Distillation: Connecting Masked Autoencoders with Contrastive Learners Bowen Shi Xiaopeng Zhang Yaoming Wang Jin Li Wenrui Dai Junni Zou H. Xiong Qi Tian 37 4 0 28 Jun 2023
Image as First-Order Norm+Linear Autoregression: Unveiling Mathematical Invariance Yinpeng Chen Xiyang Dai Dongdong Chen Mengchen Liu Lu Yuan Zicheng Liu Youzuo Lin 26 2 0 25 May 2023
What Do Self-Supervised Vision Transformers Learn? Namuk Park Wonjae Kim Byeongho Heo Taekyung Kim Sangdoo Yun SSL 67 76 1 01 May 2023
Layer Grafted Pre-training: Bridging Contrastive Learning And Masked Image Modeling For Label-Efficient Representations Ziyu Jiang Yinpeng Chen Mengchen Liu Dongdong Chen Xiyang Dai Lu Yuan Zicheng Liu Zhangyang Wang SSL VLM CLIP 32 16 0 27 Feb 2023
Self-Supervised Learning based on Heat Equation Yinpeng Chen Xiyang Dai Dongdong Chen Mengchen Liu Lu Yuan Zicheng Liu Youzuo Lin 29 4 0 23 Nov 2022
CAE v2: Context Autoencoder with CLIP Target Xinyu Zhang Jiahui Chen Junkun Yuan Qiang Chen Jian Wang ... Jimin Pi Kun Yao Junyu Han Errui Ding Jingdong Wang VLM CLIP 42 24 0 17 Nov 2022
Towards Sustainable Self-supervised Learning Shanghua Gao Pan Zhou Mingg-Ming Cheng Shuicheng Yan CLL 37 7 0 20 Oct 2022
MAR: Masked Autoencoders for Efficient Action Recognition Zhiwu Qing Shiwei Zhang Ziyuan Huang Xiang Wang Yuehuang Wang Yiliang Lv Changxin Gao Nong Sang 19 42 0 24 Jul 2022
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 305 7,434 0 11 Nov 2021
MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer Sachin Mehta Mohammad Rastegari ViT 194 1,212 0 05 Oct 2021
ResNet strikes back: An improved training procedure in timm Ross Wightman Hugo Touvron Hervé Jégou AI4TS 209 487 0 01 Oct 2021
Mobile-Former: Bridging MobileNet and Transformer Yinpeng Chen Xiyang Dai Dongdong Chen Mengchen Liu Xiaoyi Dong Lu Yuan Zicheng Liu ViT 174 476 0 12 Aug 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 308 5,773 0 29 Apr 2021
ImageNet-21K Pretraining for the Masses T. Ridnik Emanuel Ben-Baruch Asaf Noy Lihi Zelnik-Manor SSeg VLM CLIP 176 686 0 22 Apr 2021
SEED: Self-supervised Distillation For Visual Representation Zhiyuan Fang Jianfeng Wang Lijuan Wang Lei Zhang Yezhou Yang Zicheng Liu SSL 234 190 0 12 Jan 2021
Improved Baselines with Momentum Contrastive Learning Xinlei Chen Haoqi Fan Ross B. Girshick Kaiming He SSL 267 3,369 0 09 Mar 2020