CLIP-KD: An Empirical Study of CLIP Model Distillation

24 July 2023

Papers citing "CLIP-KD: An Empirical Study of CLIP Model Distillation"

29 / 29 papers shown

Title
$Simple Semi-supervised Knowledge Distillation from Vision-Language Models via $\mathbf{\texttt{D}}$ual-$\mathbf{\texttt{H}}$ead $\mathbf{\texttt{O}}$ptimization$ Simple Semi-supervised Knowledge Distillation from Vision-Language Models via $\mathbf{\texttt{D}}$ ual- $\mathbf{\texttt{H}}$ ead $\mathbf{\texttt{O}}$ ptimization Seongjae Kang Dong Bok Lee Hyungjoon Jang Sung Ju Hwang VLM 57 0 0 12 May 2025
AS3D: 2D-Assisted Cross-Modal Understanding with Semantic-Spatial Scene Graphs for 3D Visual Grounding Feng Xiao Hongbin Xu Guocan Zhao Wenxiong Kang 50 0 0 07 May 2025
A Sensitivity-Driven Expert Allocation Method in LoRA-MoE for Efficient Fine-Tuning Junzhou Xu Boyu Diao MoE 47 0 0 06 May 2025
Finger Pose Estimation for Under-screen Fingerprint Sensor Xiongjun Guan Zhiyu Pan Jianjiang Feng Jie Zhou 62 1 0 05 May 2025
AGATE: Stealthy Black-box Watermarking for Multimodal Model Copyright Protection Jianbo Gao Keke Gai Jing Yu Liehuang Zhu Qi Wu AAML 28 0 0 28 Apr 2025
Post-pre-training for Modality Alignment in Vision-Language Foundation Models Shinýa Yamaguchi Dewei Feng Sekitoshi Kanai Kazuki Adachi Daiki Chijiwa VLM 34 0 0 17 Apr 2025
Multi-party Collaborative Attention Control for Image Customization Han Yang Chuanguang Yang Qiuli Wang Zhulin An Weilun Feng Libo Huang Yongjun Xu DiffM 35 0 0 02 Apr 2025
Crossmodal Knowledge Distillation with WordNet-Relaxed Text Embeddings for Robust Image Classification Chenqi Guo Mengshuo Rong Qianli Feng Rongfan Feng Yinglong Ma VLM 68 0 0 31 Mar 2025
Efficient Continual Learning through Frequency Decomposition and Integration Ruiqi Liu Boyu Diao Libo Huang Hangda Liu Chuanguang Yang Zhulin An Yongjun Xu CLL 35 0 0 28 Mar 2025
DCA: Dividing and Conquering Amnesia in Incremental Object Detection Aoting Zhang Dongbao Yang Chang-Shu Liu Xiaopeng Hong Miao Shang Yu Zhou CLL 62 0 0 19 Mar 2025
TikZero: Zero-Shot Text-Guided Graphics Program Synthesis Jonas Belouadi Eddy Ilg M. Keuper Hideki Tanaka Masao Utiyama Raj Dabre Steffen Eger Simone Paolo Ponzetto 50 0 0 14 Mar 2025
Multi-Teacher Knowledge Distillation with Reinforcement Learning for Visual Recognition Chuanguang Yang Xinqiang Yu Han Yang Zhulin An Chengqing Yu Libo Huang Yongjun Xu 36 0 0 22 Feb 2025
Enhancing Image Generation Fidelity via Progressive Prompts Zhen Xiong Yuqi Li Chuanguang Yang Tiao Tan Zhihong Zhu Siyuan Li Yue Ma 45 1 0 13 Jan 2025
ECG-guided individual identification via PPG Riling Wei Hanjie Chen Kelu Yao Chuanguang Yang Jun Wang Chao Li 33 0 0 30 Dec 2024
MPQ-DM: Mixed Precision Quantization for Extremely Low Bit Diffusion Models Weilun Feng Haotong Qin Chuanguang Yang Zhulin An Libo Huang Boyu Diao Fei Wang Renshuai Tao Yongjun Xu Michele Magno DiffM MQ 80 5 0 16 Dec 2024
CLIP-PING: Boosting Lightweight Vision-Language Models with Proximus Intrinsic Neighbors Guidance Chu Myaet Thwal Ye Lin Tun Minh N. H. Nguyen Eui-nam Huh Choong Seon Hong VLM 74 0 0 05 Dec 2024
Swiss Army Knife: Synergizing Biases in Knowledge from Vision Foundation Models for Multi-Task Learning Yuxiang Lu Shengcao Cao Yu-xiong Wang 49 1 0 18 Oct 2024
Prototype-Driven Multi-Feature Generation for Visible-Infrared Person Re-identification Jiarui Li Zhen Qiu Yilin Yang Yuqi Li Zeyu Dong Chuanguang Yang 34 0 0 09 Sep 2024
DKDM: Data-Free Knowledge Distillation for Diffusion Models with Any Architecture Qianlong Xiang Miao Zhang Yuzhang Shang Jianlong Wu Yan Yan Liqiang Nie DiffM 63 10 0 05 Sep 2024
Zero-Shot Distillation for Image Encoders: How to Make Effective Use of Synthetic Data Niclas Popp J. H. Metzen Matthias Hein VLM 42 1 0 25 Apr 2024
Dynamic Self-adaptive Multiscale Distillation from Pre-trained Multimodal Large Model for Efficient Cross-modal Representation Learning Zhengyang Liang Meiyu Liang Wei Huang Yawen Li Zhe Xue 34 1 0 16 Apr 2024
CLIP-Embed-KD: Computationally Efficient Knowledge Distillation Using Embeddings as Teachers Lakshmi Nair VLM 31 0 0 09 Apr 2024
Mind the Modality Gap: Towards a Remote Sensing Vision-Language Model via Cross-modal Alignment Angelos Zavras Dimitrios Michail Begüm Demir Ioannis Papoutsis VLM 35 11 0 15 Feb 2024
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 305 7,443 0 11 Nov 2021
MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer Sachin Mehta Mohammad Rastegari ViT 218 1,213 0 05 Oct 2021
Mutual Contrastive Learning for Visual Representation Learning Chuanguang Yang Zhulin An Linhang Cai Yongjun Xu VLM SSL 99 75 0 26 Apr 2021
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts Soravit Changpinyo P. Sharma Nan Ding Radu Soricut VLM 290 1,084 0 17 Feb 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 301 3,708 0 11 Feb 2021
SEED: Self-supervised Distillation For Visual Representation Zhiyuan Fang Jianfeng Wang Lijuan Wang Lei Zhang Yezhou Yang Zicheng Liu SSL 239 190 0 12 Jan 2021