Zero-Shot Distillation for Image Encoders: How to Make Effective Use of
Synthetic Data

Zero-Shot Distillation for Image Encoders: How to Make Effective Use of Synthetic Data

25 April 2024

Matthias Hein

ArXiv (abs)PDF HTML

Papers citing "Zero-Shot Distillation for Image Encoders: How to Make Effective Use of Synthetic Data"

16 / 16 papers shown

Title
What Do You See? Enhancing Zero-Shot Image Classification with Multimodal Large Language Models Abdelrahman Abdelhamed Mahmoud Afifi Alec Go MLLM VLM 105 3 0 24 May 2024
Learning Vision from Models Rivals Learning Vision from Data Yonglong Tian Lijie Fan Kaifeng Chen Dina Katabi Dilip Krishnan Phillip Isola 85 51 0 28 Dec 2023
Diversify, Don't Fine-Tune: Scaling Up Visual Recognition Training with Synthetic Images Zhuoran Yu Chenchen Zhu Sean Culatana Raghuraman Krishnamoorthi Fanyi Xiao Yong Jae Lee 171 15 0 04 Dec 2023
CLIP-KD: An Empirical Study of CLIP Model Distillation Chuanguang Yang Zhulin An Libo Huang Junyu Bi Xinqiang Yu Hansheng Yang Boyu Diao Yongjun Xu VLM 100 35 0 24 Jul 2023
Synthetic Data from Diffusion Models Improves ImageNet Classification Shekoofeh Azizi Simon Kornblith Chitwan Saharia Mohammad Norouzi David J. Fleet VLM DiffM 112 315 0 17 Apr 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 432 4,656 0 30 Jan 2023
Vision-Language Pre-training: Basics, Recent Advances, and Future Trends Zhe Gan Linjie Li Chunyuan Li Lijuan Wang Zicheng Liu Jianfeng Gao VLM 65 167 0 17 Oct 2022
Knowledge Distillation as Efficient Pre-training: Faster Convergence, Higher Data-efficiency, and Better Transferability Ruifei He Shuyang Sun Jihan Yang Song Bai Xiaojuan Qi 89 37 0 10 Mar 2022
LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs Christoph Schuhmann Richard Vencu Romain Beaumont R. Kaczmarczyk Clayton Mullis Aarush Katta Theo Coombes J. Jitsev Aran Komatsuzaki VLM MLLM CLIP 243 1,444 0 03 Nov 2021
Does Knowledge Distillation Really Work? Samuel Stanton Pavel Izmailov Polina Kirichenko Alexander A. Alemi A. Wilson FedML 69 222 0 10 Jun 2021
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter Victor Sanh Lysandre Debut Julien Chaumond Thomas Wolf 267 7,554 0 02 Oct 2019
Contrastive Multiview Coding Yonglong Tian Dilip Krishnan Phillip Isola SSL 182 2,412 0 13 Jun 2019
Learning Robust Global Representations by Penalizing Local Predictive Power Haohan Wang Songwei Ge Eric Xing Zachary Chase Lipton OOD 122 967 0 29 May 2019
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks Mingxing Tan Quoc V. Le 3DV MedIm 172 18,224 0 28 May 2019
Zero-Shot Knowledge Distillation in Deep Networks Gaurav Kumar Nayak Konda Reddy Mopuri Vaisakh Shaj R. Venkatesh Babu Anirban Chakraborty 75 245 0 20 May 2019
Benchmarking Neural Network Robustness to Common Corruptions and Perturbations Dan Hendrycks Thomas G. Dietterich OOD VLM 196 3,458 0 28 Mar 2019

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.