Sensitivity of Generative VLMs to Semantically and Lexically Altered
Prompts

Sensitivity of Generative VLMs to Semantically and Lexically Altered Prompts

16 October 2024

Sri Harsha Dumpala

Chandramouli Shama Sastry

ArXiv (abs)PDF HTML

Papers citing "Sensitivity of Generative VLMs to Semantically and Lexically Altered Prompts"

15 / 15 papers shown

Title
SUGARCREPE++ Dataset: Vision-Language Model Sensitivity to Semantic and Lexical Alterations Sri Harsha Dumpala Aman Jaiswal Chandramouli Shama Sastry E. Milios Sageev Oore Hassan Sajjad CoGe 80 12 0 17 Jun 2024
Reasoning about concepts with LLMs: Inconsistencies abound Rosario A. Uceda-Sosa Karthikeyan N. Ramamurthy Maria Chang Moninder Singh 86 4 0 30 May 2024
LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion Dongfu Jiang Xiang Ren Bill Yuchen Lin ELM 82 332 0 05 Jun 2023
Consistency-guided Prompt Learning for Vision-Language Models Shuvendu Roy Ali Etemad VLM VPVLM 69 62 0 01 Jun 2023
On Evaluating Adversarial Robustness of Large Vision-Language Models Yunqing Zhao Tianyu Pang Chao Du Xiao Yang Chongxuan Li Ngai-Man Cheung Min Lin VLM AAML MLLM 122 180 0 26 May 2023
Text encoders bottleneck compositionality in contrastive vision-language models Amita Kamath Jack Hessel Kai-Wei Chang CoGe CLIP VLM 74 21 0 24 May 2023
LAION-5B: An open large-scale dataset for training next generation image-text models Christoph Schuhmann Romain Beaumont Richard Vencu Cade Gordon Ross Wightman ... Srivatsa Kundurthy Katherine Crowson Ludwig Schmidt R. Kaczmarczyk J. Jitsev VLM MLLM CLIP 200 3,493 0 16 Oct 2022
Open-Vocabulary Semantic Segmentation with Mask-adapted CLIP Feng Liang Bichen Wu Xiaoliang Dai Kunpeng Li Yinan Zhao Hang Zhang Peizhao Zhang Peter Vajda Diana Marculescu CLIP VLM 102 457 0 09 Oct 2022
MaPLe: Multi-modal Prompt Learning Muhammad Uzair Khattak H. Rasheed Muhammad Maaz Salman Khan Fahad Shahbaz Khan VPVLM VLM 256 568 0 06 Oct 2022
When and why vision-language models behave like bags-of-words, and what to do about it? Mert Yuksekgonul Federico Bianchi Pratyusha Kalluri Dan Jurafsky James Zou VLM CoGe 81 393 0 04 Oct 2022
Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models Manli Shu Weili Nie De-An Huang Zhiding Yu Tom Goldstein Anima Anandkumar Chaowei Xiao VLM VPVLM 226 304 0 15 Sep 2022
Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionality Tristan Thrush Ryan Jiang Max Bartolo Amanpreet Singh Adina Williams Douwe Kiela Candace Ross CoGe 108 427 0 07 Apr 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong Guosheng Lin MLLM BDL VLM CLIP 555 4,409 0 28 Jan 2022
FLAVA: A Foundational Language And Vision Alignment Model Amanpreet Singh Ronghang Hu Vedanuj Goswami Guillaume Couairon Wojciech Galuba Marcus Rohrbach Douwe Kiela CLIP VLM 104 715 0 08 Dec 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 967 29,810 0 26 Feb 2021