InPK: Infusing Prior Knowledge into Prompt for Vision-Language Models

27 February 2025

Shuchang Zhou

Jiwei Wei

Shiyuan He

Yuyang Zhou

Chaoning Zhang

Jie Zou

Ning Xie

Yang Yang

VLM

VPVLM

ArXiv PDF HTML

Papers citing "InPK: Infusing Prior Knowledge into Prompt for Vision-Language Models"

43 / 43 papers shown

Title
CoAPT: Context Attribute words for Prompt Tuning Gun Lee Subin An Sungyong Baik Soochahn Lee VPVLM VLM 55 1 0 18 Jul 2024
Prompt Learning via Meta-Regularization Jinyoung Park Juyeon Ko Hyunwoo J. Kim VLM VPVLM 82 16 0 01 Apr 2024
Multi-modal Attribute Prompting for Vision-Language Models Xin Liu Jiamin Wu and Wenfei Yang Xu Zhou Tianzhu Zhang VLM 52 10 0 01 Mar 2024
Concept-Guided Prompt Learning for Generalization in Vision-Language Models Yi Zhang Ce Zhang Ke Yu Yushun Tang Zhihai He VLM MLLM 63 22 0 15 Jan 2024
Simple Image-level Classification Improves Open-vocabulary Object Detection Ru Fang Guansong Pang Xiaolong Bai ObjD VLM 71 14 0 16 Dec 2023
Learning Hierarchical Prompt with Structured Linguistic Knowledge for Vision-Language Models Yubin Wang Xinyang Jiang De Cheng Dongsheng Li Cairong Zhao VLM 50 15 0 11 Dec 2023
TCP:Textual-based Class-aware Prompt tuning for Visual-Language Model Hantao Yao Rui Zhang Changsheng Xu VLM 48 29 0 30 Nov 2023
CLIP-DIY: CLIP Dense Inference Yields Open-Vocabulary Semantic Segmentation For-Free Monika Wysoczañska Michael Ramamonjisoa Tomasz Trzciñski Oriane Siméoni 3DV VLM 83 22 0 25 Sep 2023
DePT: Decoupled Prompt Tuning Ji Zhang Shihan Wu Lianli Gao Hengtao Shen Jingkuan Song VLM 52 27 0 14 Sep 2023
Distribution-Aware Prompt Tuning for Vision-Language Models Eulrang Cho Jooyeon Kim Hyunwoo J. Kim VPVLM VLM 41 22 0 06 Sep 2023
Knowledge-Aware Prompt Tuning for Generalizable Vision-Language Models Baoshuo Kan Teng Wang Wenpeng Lu Xiantong Zhen Weili Guan Feng Zheng VPVLM VLM 77 25 0 22 Aug 2023
Self-regulating Prompts: Foundational Model Adaptation without Forgetting Muhammad Uzair Khattak Syed Talal Wasim Muzammal Naseer Salman Khan Ming-Hsuan Yang Fahad Shahbaz Khan VLM 45 175 0 13 Jul 2023
Consistency-guided Prompt Learning for Vision-Language Models Shuvendu Roy Ali Etemad VLM VPVLM 40 55 0 01 Jun 2023
CM-MaskSD: Cross-Modality Masked Self-Distillation for Referring Image Segmentation Wenxuan Wang Jing Liu Xingjian He Yisi Zhang Cheng Chen Jiachen Shen Yan Zhang Jiangyun Li 39 11 0 19 May 2023
Visual-Language Prompt Tuning with Knowledge-guided Context Optimization Hantao Yao Rui Zhang Changsheng Xu VLM VPVLM 162 211 0 23 Mar 2023
Gradient-Regulated Meta-Prompt Learning for Generalizable Vision-Language Models Juncheng Li Minghe Gao Longhui Wei Siliang Tang Wenqiao Zhang Meng Li Wei Ji Qi Tian Tat-Seng Chua Yueting Zhuang VLM VPVLM 67 19 0 12 Mar 2023
Learning Customized Visual Models with Retrieval-Augmented Knowledge Haotian Liu Kilho Son Jianwei Yang Ce Liu Jianfeng Gao Yong Jae Lee Chunyuan Li VLM 91 54 0 17 Jan 2023
Visual Classification via Description from Large Language Models Sachit Menon Carl Vondrick VLM 47 297 0 13 Oct 2022
MaPLe: Multi-modal Prompt Learning Muhammad Uzair Khattak H. Rasheed Muhammad Maaz Salman Khan Fahad Shahbaz Khan VPVLM VLM 243 544 0 06 Oct 2022
Dual Modality Prompt Tuning for Vision-Language Pre-Trained Model Yinghui Xing Qirui Wu De Cheng Shizhou Zhang Guoqiang Liang Peng Wang Yanning Zhang VLM VPVLM 79 54 0 17 Aug 2022
Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classification Renrui Zhang Zhang Wei Rongyao Fang Peng Gao Kunchang Li Jifeng Dai Yu Qiao Hongsheng Li VLM 64 307 0 19 Jul 2022
Prompt-aligned Gradient for Prompt Tuning Beier Zhu Yulei Niu Yucheng Han Yuehua Wu Hanwang Zhang VLM 232 279 0 30 May 2022
Flamingo: a Visual Language Model for Few-Shot Learning Jean-Baptiste Alayrac Jeff Donahue Pauline Luc Antoine Miech Iain Barr ... Mikolaj Binkowski Ricardo Barreira Oriol Vinyals Andrew Zisserman Karen Simonyan MLLM VLM 283 3,458 0 29 Apr 2022
Visual Prompt Tuning Menglin Jia Luming Tang Bor-Chun Chen Claire Cardie Serge Belongie Bharath Hariharan Ser-Nam Lim VLM VPVLM 105 1,576 0 23 Mar 2022
Open-Vocabulary DETR with Conditional Matching Yuhang Zang Wei Li Kaiyang Zhou Chen Huang Chen Change Loy ObjD VLM 116 201 0 22 Mar 2022
Conditional Prompt Learning for Vision-Language Models Kaiyang Zhou Jingkang Yang Chen Change Loy Ziwei Liu VLM CLIP VPVLM 90 1,323 0 10 Mar 2022
Detecting Twenty-thousand Classes using Image-level Supervision Xingyi Zhou Rohit Girdhar Armand Joulin Phillip Krahenbuhl Ishan Misra CLIP VLM 87 602 0 07 Jan 2022
RegionCLIP: Region-based Language-Image Pretraining Yiwu Zhong Jianwei Yang Pengchuan Zhang Chunyuan Li Noel Codella ... Luowei Zhou Xiyang Dai Lu Yuan Yin Li Jianfeng Gao VLM CLIP 108 571 0 16 Dec 2021
Extract Free Dense Labels from CLIP Chong Zhou Chen Change Loy Bo Dai VLM CLIP 108 467 0 02 Dec 2021
Florence: A New Foundation Model for Computer Vision Lu Yuan Dongdong Chen Yi-Ling Chen Noel Codella Xiyang Dai ... Zhen Xiao Jianwei Yang Michael Zeng Luowei Zhou Pengchuan Zhang VLM 106 897 0 22 Nov 2021
FILIP: Fine-grained Interactive Language-Image Pre-Training Lewei Yao Runhu Huang Lu Hou Guansong Lu Minzhe Niu Hang Xu Xiaodan Liang Zhenguo Li Xin Jiang Chunjing Xu VLM CLIP 75 627 0 09 Nov 2021
Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling Renrui Zhang Rongyao Fang Wei Zhang Peng Gao Kunchang Li Jifeng Dai Yu Qiao Hongsheng Li VLM 254 390 0 06 Nov 2021
CLIP-Adapter: Better Vision-Language Models with Feature Adapters Peng Gao Shijie Geng Renrui Zhang Teli Ma Rongyao Fang Yongfeng Zhang Hongsheng Li Yu Qiao VLM CLIP 204 1,011 0 09 Oct 2021
Learning to Prompt for Vision-Language Models Kaiyang Zhou Jingkang Yang Chen Change Loy Ziwei Liu VPVLM CLIP VLM 443 2,340 0 02 Sep 2021
Domain Generalization: A Survey Kaiyang Zhou Ziwei Liu Yu Qiao Tao Xiang Chen Change Loy OOD AI4CE 202 999 0 03 Mar 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 681 28,659 0 26 Feb 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 392 3,778 0 11 Feb 2021
Universal Weighting Metric Learning for Cross-Modal Matching Jiwei Wei Xing Xu Yang Yang Yanli Ji Zheng Wang Heng Tao Shen 24 87 0 07 Oct 2020
EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification P. Helber B. Bischke Andreas Dengel Damian Borth 106 1,790 0 31 Aug 2017
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 453 129,831 0 12 Jun 2017
Zero-Shot Learning -- The Good, the Bad and the Ugly Yongqin Xian Bernt Schiele Zeynep Akata 60 837 0 13 Mar 2017
Describing Textures in the Wild Mircea Cimpoi Subhransu Maji Iasonas Kokkinos S. Mohamed Andrea Vedaldi 3DV 83 2,632 0 14 Nov 2013
Fine-Grained Visual Classification of Aircraft Subhransu Maji Esa Rahtu Arno Solin Matthew Blaschko Andrea Vedaldi 91 2,227 0 21 Jun 2013