Distill CLIP (DCLIP): Enhancing Image-Text Retrieval via Cross-Modal Transformer Distillation

25 May 2025

Papers citing "Distill CLIP (DCLIP): Enhancing Image-Text Retrieval via Cross-Modal Transformer Distillation"

44 / 44 papers shown

Title
TULIP: Token-length Upgraded CLIP Ivona Najdenkoska Mohammad Mahdi Derakhshani Yuki M. Asano Nanne van Noord Marcel Worring Cees G. M. Snoek VLM 80 4 0 13 Oct 2024
Selective Vision-Language Subspace Projection for Few-shot CLIP Xingyu Zhu Beier Zhu Yi Tan Shuo Wang Yanbin Hao Haiqi Zhang VLM 77 4 0 24 Jul 2024
Long-CLIP: Unlocking the Long-Text Capability of CLIP Beichen Zhang Pan Zhang Xiao-wen Dong Yuhang Zang Jiaqi Wang CLIP VLM 68 128 0 22 Mar 2024
TinyCLIP: CLIP Distillation via Affinity Mimicking and Weight Inheritance Kan Wu Houwen Peng Zhenghong Zhou Bin Xiao Mengchen Liu ... Xi Xi Chen Xinggang Wang Hongyang Chao Han Hu VLM OODD 41 60 0 21 Sep 2023
Segment Anything A. Kirillov Eric Mintun Nikhila Ravi Hanzi Mao Chloe Rolland ... Spencer Whitehead Alexander C. Berg Wan-Yen Lo Piotr Dollár Ross B. Girshick MLLM VLM 264 7,047 0 05 Apr 2023
Fine-tuned CLIP Models are Efficient Video Learners H. Rasheed Muhammad Uzair Khattak Muhammad Maaz Salman Khan Fahad Shahbaz Khan CLIP VLM 77 155 0 06 Dec 2022
CoCa: Contrastive Captioners are Image-Text Foundation Models Jiahui Yu Zirui Wang Vijay Vasudevan Legg Yeung Mojtaba Seyedhosseini Yonghui Wu VLM CLIP OffRL 127 1,286 0 04 May 2022
Masked Image Modeling Advances 3D Medical Image Analysis Zekai Chen Devansh Agarwal Kshitij Aggarwal Wiem Safta Samit Hirawat V. Sethuraman Mariann Micsinai Balan Kevin Brown 59 70 0 25 Apr 2022
ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented Visual Models Chunyuan Li Haotian Liu Liunian Harold Li Pengchuan Zhang J. Aneja ... Ping Jin Houdong Hu Zicheng Liu Yong Jae Lee Jianfeng Gao 61 148 0 19 Apr 2022
GroupViT: Semantic Segmentation Emerges from Text Supervision Jiarui Xu Shalini De Mello Sifei Liu Wonmin Byeon Thomas Breuel Jan Kautz Xinyu Wang ViT VLM 276 517 0 22 Feb 2022
Meta Knowledge Distillation Jihao Liu Boxiao Liu Hongsheng Li Yu Liu 56 26 0 16 Feb 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong Guosheng Lin MLLM BDL VLM CLIP 446 4,283 0 28 Jan 2022
RegionCLIP: Region-based Language-Image Pretraining Yiwu Zhong Jianwei Yang Pengchuan Zhang Chunyuan Li Noel Codella ... Luowei Zhou Xiyang Dai Lu Yuan Yin Li Jianfeng Gao VLM CLIP 106 571 0 16 Dec 2021
Grounded Language-Image Pre-training Liunian Harold Li Pengchuan Zhang Haotian Zhang Jianwei Yang Chunyuan Li ... Lu Yuan Lei Zhang Lei Li Kai-Wei Chang Jianfeng Gao ObjD VLM 64 1,047 0 07 Dec 2021
Florence: A New Foundation Model for Computer Vision Lu Yuan Dongdong Chen Yi-Ling Chen Noel Codella Xiyang Dai ... Zhen Xiao Jianwei Yang Michael Zeng Luowei Zhou Pengchuan Zhang VLM 106 897 0 22 Nov 2021
SimMIM: A Simple Framework for Masked Image Modeling Zhenda Xie Zheng Zhang Yue Cao Yutong Lin Jianmin Bao Zhuliang Yao Qi Dai Han Hu 154 1,331 0 18 Nov 2021
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 373 7,600 0 11 Nov 2021
CLIP-Adapter: Better Vision-Language Models with Feature Adapters Peng Gao Shijie Geng Renrui Zhang Teli Ma Rongyao Fang Yongfeng Zhang Hongsheng Li Yu Qiao VLM CLIP 204 1,011 0 09 Oct 2021
Align before Fuse: Vision and Language Representation Learning with Momentum Distillation Junnan Li Ramprasaath R. Selvaraju Akhilesh Deepak Gotmare Shafiq Joty Caiming Xiong Guosheng Lin FaML 157 1,915 0 16 Jul 2021
BEiT: BERT Pre-Training of Image Transformers Hangbo Bao Li Dong Songhao Piao Furu Wei ViT 175 2,790 0 15 Jun 2021
Knowledge distillation: A good teacher is patient and consistent Lucas Beyer Xiaohua Zhai Amelie Royer L. Markeeva Rohan Anil Alexander Kolesnikov VLM 71 292 0 09 Jun 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 587 5,920 0 29 Apr 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 681 28,659 0 26 Feb 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 392 3,778 0 11 Feb 2021
WILDS: A Benchmark of in-the-Wild Distribution Shifts Pang Wei Koh Shiori Sagawa Henrik Marklund Sang Michael Xie Marvin Zhang ... A. Kundaje Emma Pierson Sergey Levine Chelsea Finn Percy Liang OOD 146 1,418 0 14 Dec 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 400 40,217 0 22 Oct 2020
Unsupervised Learning of Visual Features by Contrasting Cluster Assignments Mathilde Caron Ishan Misra Julien Mairal Priya Goyal Piotr Bojanowski Armand Joulin OCL SSL 168 4,051 0 17 Jun 2020
End-to-End Object Detection with Transformers Nicolas Carion Francisco Massa Gabriel Synnaeve Nicolas Usunier Alexander Kirillov Sergey Zagoruyko ViT 3DV PINN 299 12,906 0 26 May 2020
Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks Xiujun Li Xi Yin Chunyuan Li Pengchuan Zhang Xiaowei Hu ... Houdong Hu Li Dong Furu Wei Yejin Choi Jianfeng Gao VLM 72 1,927 0 13 Apr 2020
Momentum Contrast for Unsupervised Visual Representation Learning Kaiming He Haoqi Fan Yuxin Wu Saining Xie Ross B. Girshick SSL 113 12,007 0 13 Nov 2019
Contrastive Representation Distillation Yonglong Tian Dilip Krishnan Phillip Isola 98 1,042 0 23 Oct 2019
LXMERT: Learning Cross-Modality Encoder Representations from Transformers Hao Hao Tan Joey Tianyi Zhou VLM MLLM 205 2,467 0 20 Aug 2019
Relational Knowledge Distillation Wonpyo Park Dongju Kim Yan Lu Minsu Cho 54 1,396 0 10 Apr 2019
Benchmarking Neural Network Robustness to Common Corruptions and Perturbations Dan Hendrycks Thomas G. Dietterich OOD VLM 107 3,399 0 28 Mar 2019
Improved Knowledge Distillation via Teacher Assistant Seyed Iman Mirzadeh Mehrdad Farajtabar Ang Li Nir Levine Akihiro Matsukawa H. Ghasemzadeh 79 1,073 0 09 Feb 2019
Representation Learning with Contrastive Predictive Coding Aaron van den Oord Yazhe Li Oriol Vinyals DRL SSL 231 10,152 0 10 Jul 2018
Born Again Neural Networks Tommaso Furlanello Zachary Chase Lipton Michael Tschannen Laurent Itti Anima Anandkumar 63 1,030 0 12 May 2018
Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer Sergey Zagoruyko N. Komodakis 101 2,561 0 12 Dec 2016
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations Ranjay Krishna Yuke Zhu Oliver Groth Justin Johnson Kenji Hata ... Yannis Kalantidis Li Li David A. Shamma Michael S. Bernstein Fei-Fei Li 170 5,706 0 23 Feb 2016
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks Shaoqing Ren Kaiming He Ross B. Girshick Jian Sun AIMat ObjD 412 61,900 0 04 Jun 2015
Distilling the Knowledge in a Neural Network Geoffrey E. Hinton Oriol Vinyals J. Dean FedML 241 19,523 0 09 Mar 2015
FitNets: Hints for Thin Deep Nets Adriana Romero Nicolas Ballas Samira Ebrahimi Kahou Antoine Chassang C. Gatta Yoshua Bengio FedML 236 3,862 0 19 Dec 2014
Deep Visual-Semantic Alignments for Generating Image Descriptions A. Karpathy Li Fei-Fei 60 5,569 0 07 Dec 2014
Microsoft COCO: Common Objects in Context Nayeon Lee Michael Maire Serge J. Belongie Lubomir Bourdev Ross B. Girshick James Hays Pietro Perona Deva Ramanan C. L. Zitnick Piotr Dollár ObjD 272 43,290 0 01 May 2014