v1v2v3v4v5 (latest)

SCAM: A Real-World Typographic Robustness Evaluation for Multimodal Foundation Models

7 April 2025

Papers citing "SCAM: A Real-World Typographic Robustness Evaluation for Multimodal Foundation Models"

40 / 40 papers shown

Title
SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features Michael Tschannen A. Gritsenko Xiao Wang Muhammad Ferjad Naeem Ibrahim Alabdulmohsin ... Basil Mustafa Olivier J. Hénaff Jeremiah Harmsen Andreas Steiner Xiaohua Zhai VLM 132 79 0 21 Feb 2025
Does Prompt Formatting Have Any Impact on LLM Performance? Jia He Mukund Rungta David Koleczek Arshdeep Sekhon Franklin X Wang Sadid Hasan LLMAG LRM 82 57 0 15 Nov 2024
GPT-4o System Card OpenAI OpenAI : Aaron Hurst Adam Lerer Adam P. Goucher ... Yuchen He Yuchen Zhang Yujia Jin Yunxing Dai Yury Malkov MLLM 202 1,019 0 25 Oct 2024
Empirical Analysis of Large Vision-Language Models against Goal Hijacking via Visual Prompt Injection Subaru Kimura Ryota Tanaka Shumpei Miyawaki Jun Suzuki Keisuke Sakaguchi MLLM 66 7 0 07 Aug 2024
On the Worst Prompt Performance of Large Language Models Bowen Cao Deng Cai Zhisong Zhang Yuexian Zou Wai Lam ALM LRM 68 8 0 08 Jun 2024
Typography Leads Semantic Diversifying: Amplifying Adversarial Transferability across Multimodal Large Language Models Hao-Ran Cheng Erjia Xiao Jiahang Cao Le Yang Kaidi Xu Jindong Gu Renjing Xu AAML 118 10 0 30 May 2024
Towards Transferable Attacks Against Vision-LLMs in Autonomous Driving with Typography N. Chung Sensen Gao Tuan-Anh Vu Jie M. Zhang Aishan Liu Yun Lin Jin Song Dong Qi Guo AAML 92 11 0 23 May 2024
Understanding Retrieval-Augmented Task Adaptation for Vision-Language Models Yifei Ming Yixuan Li VLM 110 8 0 02 May 2024
Vision-Language Models for Medical Report Generation and Visual Question Answering: A Review Iryna Hartsock Ghulam Rasool 87 79 0 04 Mar 2024
Unveiling Typographic Deceptions: Insights of the Typographic Vulnerability in Large Vision-Language Model Hao-Ran Cheng Erjia Xiao Jindong Gu Le Yang Jinhao Duan Jize Zhang Jiahang Cao Kaidi Xu Renjing Xu 72 9 0 29 Feb 2024
Vision-LLMs Can Fool Themselves with Self-Generated Typographic Attacks Maan Qraitem Nazia Tasnim Piotr Teterwak Kate Saenko Bryan A. Plummer AAML VLM 76 12 0 01 Feb 2024
State of What Art? A Call for Multi-Prompt LLM Evaluation Moran Mizrahi Guy Kaplan Daniel Malkin Rotem Dror Dafna Shahaf Gabriel Stanovsky ELM 91 147 0 31 Dec 2023
CLIP in Medical Imaging: A Comprehensive Survey Zihao Zhao Yuxiao Liu Han Wu Yonghao Li Sheng Wang L. Teng Disheng Liu Zhiming Cui Qian Wang Dinggang Shen CLIP MedIm LM&MA VLM 91 43 0 12 Dec 2023
FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts Yichen Gong Delong Ran Jinyuan Liu Conglei Wang Tianshuo Cong Anyu Wang Sisi Duan Xiaoyun Wang MLLM 218 158 0 09 Nov 2023
Vision Language Models in Autonomous Driving: A Survey and Outlook Xingcheng Zhou Mingyu Liu Ekim Yurtsever B. L. Žagar Walter Zimmer Hu Cao Alois C. Knoll VLM 70 59 0 22 Oct 2023
Improved Baselines with Visual Instruction Tuning Haotian Liu Chunyuan Li Yuheng Li Yong Jae Lee VLM MLLM 130 2,807 0 05 Oct 2023
Kandinsky: an Improved Text-to-Image Synthesis with Image Prior and Latent Diffusion Anton Razzhigaev Arseniy Shakhmatov Anastasia Maltseva V.Ya. Arkhipkin Igor Pavlov Ilya Ryabov Angelina Kuts Alexander Panchenko Andrey Kuznetsov Denis Dimitrov 108 81 0 05 Oct 2023
NLLB-CLIP -- train performant multilingual image retrieval model on a budget Alexander Visheratin VLM 86 19 0 04 Sep 2023
LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day Chunyuan Li Cliff Wong Sheng Zhang Naoto Usuyama Haotian Liu Jianwei Yang Tristan Naumann Hoifung Poon Jianfeng Gao LM&MA MedIm 116 792 0 01 Jun 2023
DataComp: In search of the next generation of multimodal datasets S. Gadre Gabriel Ilharco Alex Fang J. Hayase Georgios Smyrnis ... A. Dimakis J. Jitsev Y. Carmon Vaishaal Shankar Ludwig Schmidt VLM 92 447 0 27 Apr 2023
Visual Instruction Tuning Haotian Liu Chunyuan Li Qingyang Wu Yong Jae Lee SyDa VLM MLLM 562 4,910 0 17 Apr 2023
Defense-Prefix for Preventing Typographic Attacks on CLIP Hiroki Azuma Yusuke Matsui VLM AAML 62 19 0 10 Apr 2023
Sigmoid Loss for Language Image Pre-Training Xiaohua Zhai Basil Mustafa Alexander Kolesnikov Lucas Beyer CLIP VLM 232 1,200 0 27 Mar 2023
Reproducible scaling laws for contrastive language-image learning Mehdi Cherti Romain Beaumont Ross Wightman Mitchell Wortsman Gabriel Ilharco Cade Gordon Christoph Schuhmann Ludwig Schmidt J. Jitsev VLM CLIP 118 818 0 14 Dec 2022
ZegCLIP: Towards Adapting CLIP for Zero-shot Semantic Segmentation Ziqi Zhou Bowen Zhang Yinjie Lei Lingqiao Liu Yifan Liu VLM 74 175 0 07 Dec 2022
CLIP-Nav: Using CLIP for Zero-Shot Vision-and-Language Navigation Vishnu Sashank Dorbala Gunnar Sigurdsson Robinson Piramuthu Jesse Thomason Gaurav Sukhatme LM&Ro 84 56 0 30 Nov 2022
Vision-Language Pre-training: Basics, Recent Advances, and Future Trends Zhe Gan Linjie Li Chunyuan Li Lijuan Wang Zicheng Liu Jianfeng Gao VLM 58 165 0 17 Oct 2022
LAION-5B: An open large-scale dataset for training next generation image-text models Christoph Schuhmann Romain Beaumont Richard Vencu Cade Gordon Ross Wightman ... Srivatsa Kundurthy Katherine Crowson Ludwig Schmidt R. Kaczmarczyk J. Jitsev VLM MLLM CLIP 200 3,493 0 16 Oct 2022
Visual Classification via Description from Large Language Models Sachit Menon Carl Vondrick VLM 64 302 0 13 Oct 2022
Patching open-vocabulary models by interpolating weights Gabriel Ilharco Mitchell Wortsman S. Gadre Shuran Song Hannaneh Hajishirzi Simon Kornblith Ali Farhadi Ludwig Schmidt VLM KELM 104 176 0 10 Aug 2022
Disentangling visual and written concepts in CLIP Joanna Materzyñska Antonio Torralba David Bau CoGe 73 51 0 15 Jun 2022
Hierarchical Text-Conditional Image Generation with CLIP Latents Aditya A. Ramesh Prafulla Dhariwal Alex Nichol Casey Chu Mark Chen VLM DiffM 413 6,897 0 13 Apr 2022
Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors Oran Gafni Adam Polyak Oron Ashual Shelly Sheynin Devi Parikh Yaniv Taigman DiffM 79 523 0 24 Mar 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong Guosheng Lin MLLM BDL VLM CLIP 547 4,398 0 28 Jan 2022
Cross Modal Retrieval with Querybank Normalisation Simion-Vlad Bogolin Ioana Croitoru Hailin Jin Yang Liu Samuel Albanie 66 87 0 23 Dec 2021
High-Resolution Image Synthesis with Latent Diffusion Models Robin Rombach A. Blattmann Dominik Lorenz Patrick Esser Bjorn Ommer 3DV 474 15,734 0 20 Dec 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 967 29,810 0 26 Feb 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 453 3,887 0 11 Feb 2021
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 670 41,369 0 22 Oct 2020
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 2.2K 194,322 0 10 Dec 2015