Understanding and Mitigating Overfitting in Prompt Tuning for Vision-Language Models

4 November 2022

Yang Liu

Papers citing "Understanding and Mitigating Overfitting in Prompt Tuning for Vision-Language Models"

31 / 31 papers shown

Title
Prompting Video-Language Foundation Models with Domain-specific Fine-grained Heuristics for Video Question Answering Ting Yu Kunhao Fu Shuhui Wang Qingming Huang Jun Yu 46 0 0 12 Oct 2024
Language-Assisted Human Part Motion Learning for Skeleton-Based Temporal Action Segmentation Bowen Chen Haoyu Ji Zhiyong Wang Benjamin Filtjens C. Wang Weihong Ren Bart Vanrumste Honghai Liu 53 0 0 08 Oct 2024
Craft: Cross-modal Aligned Features Improve Robustness of Prompt Tuning Jingchen Sun Rohan Sharma Vishnu Suresh Lokhande Changyou Chen 41 0 0 22 Jul 2024
Exploring Scalability of Self-Training for Open-Vocabulary Temporal Action Localization Jeongseok Hyun Su Ho Han Hyolim Kang Joon-Young Lee Seon Joo Kim VLM 42 2 0 09 Jul 2024
Learning Visual Conditioning Tokens to Correct Domain Shift for Fully Test-time Adaptation Yushun Tang Shuoshuo Chen Zhehan Kan Yi Zhang Qinghai Guo Zhihai He 51 2 0 27 Jun 2024
Harmonizing Generalization and Personalization in Federated Prompt Learning Tianyu Cui Hongxia Li Jingya Wang Ye-ling Shi FedML VLM 34 8 0 16 May 2024
AAPL: Adding Attributes to Prompt Learning for Vision-Language Models Gahyeon Kim Sohee Kim Seokju Lee VLM 33 5 0 25 Apr 2024
Compositional Kronecker Context Optimization for Vision-Language Models Kun Ding Xiaohui Li Qiang Yu Ying Wang Haojian Zhang Shiming Xiang VLM 44 0 0 18 Mar 2024
Language-Driven Visual Consensus for Zero-Shot Semantic Segmentation Zicheng Zhang Tong Zhang Yi Zhu Jian-zhuo Liu Xiaodan Liang QiXiang Ye Wei Ke VLM 49 2 0 13 Mar 2024
One Prompt Word is Enough to Boost Adversarial Robustness for Pre-trained Vision-Language Models Lin Li Haoyan Guan Jianing Qiu Michael W. Spratling AAML VLM VPVLM 31 21 0 04 Mar 2024
OV-NeRF: Open-vocabulary Neural Radiance Fields with Vision and Language Foundation Models for 3D Semantic Understanding Guibiao Liao Kaichen Zhou Zhenyu Bao Kanglin Liu Qing Li VLM 20 20 0 07 Feb 2024
Learning to Prompt Segment Anything Models Jiaxing Huang Kai Jiang Jingyi Zhang Han Qiu Lewei Lu Shijian Lu Eric P. Xing VLM LRM 48 7 0 09 Jan 2024
Few-shot Adaptation of Multi-modal Foundation Models: A Survey Fan Liu Tianshu Zhang Wenwen Dai Wenwen Cai Wenwen Cai Xiaocong Zhou Delong Chen VLM OffRL 31 23 0 03 Jan 2024
Domain Aligned CLIP for Few-shot Classification Muhammad Waleed Gondal Jochen Gast Inigo Alonso Ruiz Richard Droste Tommaso Macri Suren Kumar Luitpold Staudigl VLM 21 11 0 15 Nov 2023
Meta-Adapter: An Online Few-shot Learner for Vision-Language Model Cheng Cheng Lin Song Ruoyi Xue Hang Wang Hongbin Sun Yixiao Ge Ying Shan VLM ObjD 39 18 0 07 Nov 2023
Integrating Language-Derived Appearance Elements with Visual Cues in Pedestrian Detection Sungjune Park Hyunjun Kim Y. Ro 45 11 0 02 Nov 2023
Tuning Multi-mode Token-level Prompt Alignment across Modalities Dongsheng Wang Miaoge Li Xinyang Liu Mingsheng Xu Bo Chen Hanwang Zhang VLM 39 16 0 25 Sep 2023
DualCoOp++: Fast and Effective Adaptation to Multi-Label Recognition with Limited Annotations Ping Hu Ximeng Sun Stan Sclaroff Kate Saenko VLM 34 21 0 03 Aug 2023
DVPT: Dynamic Visual Prompt Tuning of Large Pre-trained Models for Medical Image Analysis Along He Kai Wang Zhihong Wang Tao Li Huazhu Fu MedIm 25 3 0 19 Jul 2023
UniBoost: Unsupervised Unimodal Pre-training for Boosting Zero-shot Vision-Language Tasks Yanan Sun Zi-Qi Zhong Qi Fan Chi-Keung Tang Yu-Wing Tai VLM 33 4 0 07 Jun 2023
Visual Tuning Bruce X. B. Yu Jianlong Chang Haixin Wang Lin Liu Shijie Wang ... Lingxi Xie Haojie Li Zhouchen Lin Qi Tian Chang Wen Chen VLM 49 38 0 10 May 2023
Vision-Language Models for Vision Tasks: A Survey Jingyi Zhang Jiaxing Huang Sheng Jin Shijian Lu VLM 41 483 0 03 Apr 2023
Patch-Prompt Aligned Bayesian Prompt Tuning for Vision-Language Models Xinyang Liu Dongsheng Wang Bowei Fang Miaoge Li Zhibin Duan Yishi Xu Bo Chen Mingyuan Zhou VLM VPVLM 29 5 0 16 Mar 2023
Pro-tuning: Unified Prompt Tuning for Vision Tasks Xing Nie Bolin Ni Jianlong Chang Gaomeng Meng Chunlei Huo Zhaoxiang Zhang Shiming Xiang Qi Tian Chunhong Pan AAML VPVLM VLM 32 69 0 28 Jul 2022
Prompt-aligned Gradient for Prompt Tuning Beier Zhu Yulei Niu Yucheng Han Yuehua Wu Hanwang Zhang VLM 186 271 0 30 May 2022
Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling Renrui Zhang Rongyao Fang Wei Zhang Peng Gao Kunchang Li Jifeng Dai Yu Qiao Hongsheng Li VLM 192 385 0 06 Nov 2021
CPT: Colorful Prompt Tuning for Pre-trained Vision-Language Models Yuan Yao Ao Zhang Zhengyan Zhang Zhiyuan Liu Tat-Seng Chua Maosong Sun MLLM VPVLM VLM 208 221 0 24 Sep 2021
Learning to Prompt for Vision-Language Models Kaiyang Zhou Jingkang Yang Chen Change Loy Ziwei Liu VPVLM CLIP VLM 348 2,271 0 02 Sep 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 280 3,848 0 18 Apr 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 310 3,708 0 11 Feb 2021
Free Lunch for Few-shot Learning: Distribution Calibration Shuo Yang Lu Liu Min Xu OODD 219 322 0 16 Jan 2021