Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models

15 September 2022

De-An Huang

Papers citing "Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models"

46 / 46 papers shown

Title
Diff-Prompt: Diffusion-Driven Prompt Generator with Mask Supervision Weicai Yan Wang Lin Zirun Guo Ye Wang Fangming Feng Xiaoda Yang Z. Wang Tao Jin DiffM 120 2 0 30 Apr 2025
JailDAM: Jailbreak Detection with Adaptive Memory for Vision-Language Model Yi Nian Shenzhe Zhu Yuehan Qin Li Li Z. Wang Chaowei Xiao Yue Zhao 28 0 0 03 Apr 2025
Mitigating Cache Noise in Test-Time Adaptation for Large Vision-Language Models Haotian Zhai Xinyu Chen Can Zhang Tianming Sha Ruirui Li BDL VLM 67 0 0 24 Mar 2025
FisherTune: Fisher-Guided Robust Tuning of Vision Foundation Models for Domain Generalized Segmentation Dong Zhao Jinlong Li Shuang Wang Mengyao Wu Qi Zang N. Sebe Zhun Zhong 141 0 0 23 Mar 2025
Point-Cache: Test-time Dynamic and Hierarchical Cache for Robust and Generalizable Point Cloud Analysis Hongyu Sun Qiuhong Ke Ming Cheng Y. Wang Deying Li Chenhui Gou Jianfei Cai 3DPC 92 0 0 15 Mar 2025
ProAPO: Progressively Automatic Prompt Optimization for Visual Classification Xiangyan Qu Gaopeng Gou Jiamin Zhuang Jing Yu Kun Song Qihao Wang Yili Li Gang Xiong VLM 83 0 0 13 Mar 2025
Generalizable Prompt Learning of CLIP: A Brief Overview Fangming Cui Yonggang Zhang Xuan Wang Xule Wang Liang Xiao VPVLM VLM 150 0 0 03 Mar 2025
Noise is an Efficient Learner for Zero-Shot Vision-Language Models Raza Imam Asif Hanif Jian Zhang Khaled Waleed Dawoud Yova Kementchedjhieva Mohammad Yaqub VLM 53 0 0 09 Feb 2025
Hummingbird: High Fidelity Image Generation via Multimodal Context Alignment Minh-Quan Le Gaurav Mittal Tianjian Meng A S M Iftekhar Vishwas Suryanarayanan Barun Patra Dimitris Samaras Mei Chen DiffM 62 0 0 07 Feb 2025
ProKeR: A Kernel Perspective on Few-Shot Adaptation of Large Vision-Language Models Yassir Bendou Amine Ouasfi Vincent Gripon A. Boukhayma VLM 51 0 0 19 Jan 2025
Words Matter: Leveraging Individual Text Embeddings for Code Generation in CLIP Test-Time Adaptation Shambhavi Mishra Julio Silva-Rodrıguez Ismail ben Ayed M. Pedersoli Jose Dolz VLM 79 1 0 26 Nov 2024
Sensitivity of Generative VLMs to Semantically and Lexically Altered Prompts Sri Harsha Dumpala Aman Jaiswal Chandramouli Shama Sastry E. Milios Sageev Oore Hassan Sajjad VLM 23 2 0 16 Oct 2024
How Does Diverse Interpretability of Textual Prompts Impact Medical Vision-Language Zero-Shot Tasks? Sicheng Wang Che Liu Rossella Arcucci VLM MedIm 36 0 0 31 Aug 2024
ControlMLLM: Training-Free Visual Prompt Learning for Multimodal Large Language Models Ming-Kuan Wu Xinyue Cai Jiayi Ji Jiale Li Oucheng Huang Gen Luo Hao Fei Xiaoshuai Sun Rongrong Ji MLLM 45 7 0 31 Jul 2024
AWT: Transferring Vision-Language Models via Augmentation, Weighting, and Transportation Yuhan Zhu Yuyang Ji Zhiyu Zhao Gangshan Wu Limin Wang VLM 39 7 0 05 Jul 2024
SAFT: Towards Out-of-Distribution Generalization in Fine-Tuning Bac Nguyen Stefan Uhlich Fabien Cardinaux Lukas Mauch Marzieh Edraki Aaron Courville OODD CLL VLM 54 3 0 03 Jul 2024
Language-guided Detection and Mitigation of Unknown Dataset Bias Zaiying Zhao Soichiro Kumano Toshihiko Yamasaki 36 2 0 05 Jun 2024
On the test-time zero-shot generalization of vision-language models: Do we really need prompt learning? Maxime Zanella Ismail Ben Ayed VLM MLLM 46 22 0 03 May 2024
Test-Time Model Adaptation with Only Forward Passes Shuaicheng Niu Chunyan Miao Guohao Chen Pengcheng Wu Peilin Zhao TTA 38 18 0 02 Apr 2024
Unknown Prompt, the only Lacuna: Unveiling CLIP's Potential for Open Domain Generalization Mainak Singha Ankit Jha Shirsha Bose Ashwin Nair Moloud Abdar Biplab Banerjee VLM 39 10 0 31 Mar 2024
Just Shift It: Test-Time Prototype Shifting for Zero-Shot Generalization with Vision-Language Models Elaine Sui Xiaohan Wang Serena Yeung-Levy VLM 30 5 0 19 Mar 2024
Enhancing Vision-Language Pre-training with Rich Supervisions Yuan Gao Kunyu Shi Pengkai Zhu Edouard Belval Oren Nuriel Srikar Appalaraju Shabnam Ghadar Vijay Mahadevan Zhuowen Tu Stefano Soatto VLM CLIP 62 12 0 05 Mar 2024
Spurious Feature Eraser: Stabilizing Test-Time Adaptation for Vision-Language Foundation Model Huan Ma Yan Zhu Changqing Zhang Peilin Zhao Baoyuan Wu Long-Kai Huang Qinghua Hu Bing Wu VLM 67 1 0 01 Mar 2024
HGCLIP: Exploring Vision-Language Models with Graph Representations for Hierarchical Understanding Peng Xia Xingtong Yu Ming Hu Lie Ju Zhiyong Wang Peibo Duan Zongyuan Ge VLM 45 9 0 23 Nov 2023
Zero-Shot and Few-Shot Video Question Answering with Multi-Modal Prompts Bipin Rajendran Bashir M. Al-Hashimi MLLM VLM 30 2 0 27 Sep 2023
CWCL: Cross-Modal Transfer with Continuously Weighted Contrastive Loss R. S. Srinivasa Jaejin Cho Chouchang Yang Yashas Malur Saidutta Ching Hua Lee Yilin Shen Hongxia Jin VLM 29 8 0 26 Sep 2023
SCT: A Simple Baseline for Parameter-Efficient Fine-Tuning via Salient Channels Henry Hengyuan Zhao Pichao Wang Yuyang Zhao Hao Luo F. Wang Mike Zheng Shou ViT 34 14 0 15 Sep 2023
Bootstrap Fine-Grained Vision-Language Alignment for Unified Zero-Shot Anomaly Localization Hanqiu Deng Zhaoxiang Zhang Jinan Bao Xingyu Li VLM 27 4 0 30 Aug 2023
Cross-Modal Retrieval Meets Inference:Improving Zero-Shot Classification with Cross-Modal Retrieval Seong-Hoon Eom Namgyu Ho Jaehoon Oh Se-Young Yun CLIP VLM 31 0 0 29 Aug 2023
A Foundation Language-Image Model of the Retina (FLAIR): Encoding Expert Knowledge in Text Supervision Julio Silva-Rodríguez H. Chakor Riadh Kobbi Jose Dolz Ismail Ben Ayed VLM MedIm 64 33 0 15 Aug 2023
Prompt Pre-Training with Twenty-Thousand Classes for Open-Vocabulary Visual Recognition Shuhuai Ren Aston Zhang Yi Zhu Shuai Zhang Shuai Zheng Mu Li Alexander J. Smola Xu Sun VPVLM VLM 21 28 0 10 Apr 2023
Defense-Prefix for Preventing Typographic Attacks on CLIP Hiroki Azuma Yusuke Matsui VLM AAML 18 17 0 10 Apr 2023
Robust Test-Time Adaptation in Dynamic Scenarios Longhui Yuan Binhui Xie Shuangliang Li TTA 27 109 0 24 Mar 2023
A Simple Zero-shot Prompt Weighting Technique to Improve Prompt Ensembling in Text-Image Models J. Allingham Jie Jessie Ren Michael W. Dusenberry Xiuye Gu Yin Cui Dustin Tran J. Liu Balaji Lakshminarayanan LLMAG VLM 26 32 0 13 Feb 2023
CLIPood: Generalizing CLIP to Out-of-Distributions Yang Shu Xingzhuo Guo Jialong Wu Ximei Wang Jianmin Wang Mingsheng Long OODD VLM 41 74 0 02 Feb 2023
Understanding Zero-Shot Adversarial Robustness for Large-Scale Models Chengzhi Mao Scott Geng Junfeng Yang Xin Eric Wang Carl Vondrick VLM 34 59 0 14 Dec 2022
Improving Zero-shot Generalization and Robustness of Multi-modal Models Yunhao Ge Jie Jessie Ren Andrew Gallagher Yuxiao Wang Ming Yang Hartwig Adam Laurent Itti Balaji Lakshminarayanan Jiaping Zhao VLM 24 33 0 04 Dec 2022
MaPLe: Multi-modal Prompt Learning Muhammad Uzair Khattak H. Rasheed Muhammad Maaz Salman Khan F. Khan VPVLM VLM 188 530 0 06 Oct 2022
Prompt-aligned Gradient for Prompt Tuning Beier Zhu Yulei Niu Yucheng Han Yuehua Wu Hanwang Zhang VLM 181 271 0 30 May 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong S. Hoi MLLM BDL VLM CLIP 390 4,125 0 28 Jan 2022
Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling Renrui Zhang Rongyao Fang Wei Zhang Peng Gao Kunchang Li Jifeng Dai Yu Qiao Hongsheng Li VLM 189 385 0 06 Nov 2021
Learning to Prompt for Vision-Language Models Kaiyang Zhou Jingkang Yang Chen Change Loy Ziwei Liu VPVLM CLIP VLM 325 2,263 0 02 Sep 2021
Towards Out-Of-Distribution Generalization: A Survey Jiashuo Liu Zheyan Shen Yue He Xingxuan Zhang Renzhe Xu Han Yu Peng Cui CML OOD 31 515 0 31 Aug 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 280 3,844 0 18 Apr 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 298 3,693 0 11 Feb 2021
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks Chelsea Finn Pieter Abbeel Sergey Levine OOD 314 11,681 0 09 Mar 2017