On Evaluating Adversarial Robustness of Large Vision-Language Models

26 May 2023

Papers citing "On Evaluating Adversarial Robustness of Large Vision-Language Models"

50 / 136 papers shown

Title
DocVXQA: Context-Aware Visual Explanations for Document Question Answering Mohamed Ali Souibgui Changkyu Choi Andrey Barsky Kangsoo Jung Ernest Valveny Dimosthenis Karatzas 28 0 0 12 May 2025
X-Transfer Attacks: Towards Super Transferable Adversarial Attacks on CLIP Hanxun Huang Sarah Monazam Erfani Yige Li Xingjun Ma James Bailey AAML 44 0 0 08 May 2025
Adversarial Robustness Analysis of Vision-Language Models in Medical Image Segmentation Anjila Budathoki Manish Dhakal AAML 39 0 0 05 May 2025
Transferable Adversarial Attacks on Black-Box Vision-Language Models Kai Hu Weichen Yu L. Zhang Alexander Robey Andy Zou Chengming Xu Haoqi Hu Matt Fredrikson AAML VLM 64 0 0 02 May 2025
Hydra: An Agentic Reasoning Approach for Enhancing Adversarial Robustness and Mitigating Hallucinations in Vision-Language Models Chung-En Hsuan-Chih Chen Brian Jalaian Nathaniel D. Bastian AAML VLM 44 0 0 19 Apr 2025
Manipulating Multimodal Agents via Cross-Modal Prompt Injection Le Wang Zonghao Ying Tianyuan Zhang Siyuan Liang Shengshan Hu Mingchuan Zhang A. Liu Xianglong Liu AAML 33 1 0 19 Apr 2025
COUNTS: Benchmarking Object Detectors and Multimodal Large Language Models under Distribution Shifts Jiansheng Li Xingxuan Zhang Hao Zou Yige Guo Renzhe Xu Yilong Liu Chuzhao Zhu Yue He Peng Cui VLM 42 0 0 14 Apr 2025
A Domain-Based Taxonomy of Jailbreak Vulnerabilities in Large Language Models Carlos Peláez-González Andrés Herrera-Poyatos Cristina Zuheros David Herrera-Poyatos Virilo Tejedor F. Herrera AAML 24 0 0 07 Apr 2025
On the Robustness of GUI Grounding Models Against Image Attacks Haoren Zhao Tianyi Chen Zhen Wang AAML 36 0 0 07 Apr 2025
AdPO: Enhancing the Adversarial Robustness of Large Vision-Language Models with Preference Optimization Chaohu Liu Tianyi Gui Yu Liu Linli Xu VLM AAML 68 1 0 02 Apr 2025
Safeguarding Vision-Language Models: Mitigating Vulnerabilities to Gaussian Noise in Perturbation-based Attacks Jiawei Wang Yushen Zuo Yuanjun Chai Ziqiang Liu Yichen Fu Yichun Feng Kin-Man Lam AAML VLM 42 0 0 02 Apr 2025
How does Watermarking Affect Visual Language Models in Document Understanding? Chunxue Xu Yiwei Wang Bryan Hooi Yujun Cai Songze Li VLM 44 0 0 01 Apr 2025
Protecting Your Video Content: Disrupting Automated Video-based LLM Annotations Haitong Liu Kuofeng Gao Yang Bai Jinmin Li Jinxiao Shan Tao Dai Shu-Tao Xia AAML 65 1 0 26 Mar 2025
MIRAGE: Multimodal Immersive Reasoning and Guided Exploration for Red-Team Jailbreak Attacks Wenhao You Bryan Hooi Yiwei Wang Yixuan Wang Zong Ke Ming Yang Zi Huang Yujun Cai AAML 58 0 0 24 Mar 2025
Survey of Adversarial Robustness in Multimodal Large Language Models Chengze Jiang Zhuangzhuang Wang Minjing Dong Jie Gui AAML 63 0 0 18 Mar 2025
Evolution-based Region Adversarial Prompt Learning for Robustness Enhancement in Vision-Language Models Xiaojun Jia Sensen Gao Simeng Qin Ke Ma Xianrui Li Yihao Huang Wei Dong Yang Liu Xiaochun Cao AAML VLM 60 0 0 17 Mar 2025
Making Every Step Effective: Jailbreaking Large Vision-Language Models Through Hierarchical KV Equalization Shuyang Hao Yiwei Wang Bryan Hooi Jiaheng Liu Muhao Chen Zi Huang Yujun Cai AAML VLM 67 0 0 14 Mar 2025
Safety Mirage: How Spurious Correlations Undermine VLM Safety Fine-tuning Yiwei Chen Yuguang Yao Yihua Zhang Bingquan Shen Gaowen Liu Sijia Liu AAML MU 58 1 0 14 Mar 2025
Attacking Multimodal OS Agents with Malicious Image Patches Lukas Aichberger Alasdair Paren Y. Gal Philip H. S. Torr Adel Bibi AAML 59 2 0 13 Mar 2025
A Frustratingly Simple Yet Highly Effective Attack Baseline: Over 90% Success Rate Against the Strong Black-box Models of GPT-4.5/4o/o1 Zhaoyi Li Xiaohan Zhao Dong-Dong Wu Jiacheng Cui Zhiqiang Shen AAML VLM 75 0 0 13 Mar 2025
Poisoned-MRAG: Knowledge Poisoning Attacks to Multimodal Retrieval Augmented Generation Yinuo Liu Zenghui Yuan Guiyao Tie Jiawen Shi Lichao Sun Lichao Sun Neil Zhenqiang Gong 46 1 0 08 Mar 2025
CeTAD: Towards Certified Toxicity-Aware Distance in Vision Language Models Xiangyu Yin Jiaxu Liu Zhen Chen Jinwei Hu Yi Dong Xiaowei Huang Wenjie Ruan AAML 47 0 0 08 Mar 2025
CLIP is Strong Enough to Fight Back: Test-time Counterattacks towards Zero-shot Adversarial Robustness of CLIP Songlong Xing Zhengyu Zhao N. Sebe AAML 62 0 0 05 Mar 2025
SafeVLA: Towards Safety Alignment of Vision-Language-Action Model via Safe Reinforcement Learning Borong Zhang Yuhao Zhang Yalan Qin Yingshan Lei Josef Dai Yuanpei Chen Yaodong Yang 66 4 0 05 Mar 2025
Words or Vision: Do Vision-Language Models Have Blind Faith in Text? Ailin Deng Tri Cao Zhirui Chen Bryan Hooi VLM 98 2 0 04 Mar 2025
FC-Attack: Jailbreaking Large Vision-Language Models via Auto-Generated Flowcharts Ziyi Zhang Zhen Sun Zhe Zhang Jihui Guo Xinlei He AAML 52 2 0 28 Feb 2025
Improving Adversarial Transferability in MLLMs via Dynamic Vision-Language Alignment Attack Chenhe Gu Jindong Gu Andong Hua Yao Qin AAML 47 0 0 27 Feb 2025
Stealthy Backdoor Attack in Self-Supervised Learning Vision Encoders for Large Vision Language Models Zhaoyi Liu Huan Zhang AAML 86 0 0 25 Feb 2025
EigenShield: Causal Subspace Filtering via Random Matrix Theory for Adversarially Robust Vision-Language Models Nastaran Darabi Devashri Naik Sina Tayebati Dinithi Jayasuriya Ranganath Krishnan A. R. Trivedi AAML 52 0 0 24 Feb 2025
On the Robustness of Transformers against Context Hijacking for Linear Classification Tianle Li Chenyang Zhang Xingwu Chen Yuan Cao Difan Zou 72 0 0 24 Feb 2025
Testing the limits of fine-tuning to improve reasoning in vision language models Luca M. Schulze Buschoff Konstantinos Voudouris Elif Akata Matthias Bethge Joshua B. Tenenbaum Eric Schulz LRM VLM Presented at ResearchTrend Connect \| VLM on 14 Mar 2025 124 0 1 24 Feb 2025
Tracking the Copyright of Large Vision-Language Models through Parameter Learning Adversarial Images Yubo Wang Jianting Tang Chaohu Liu Linli Xu AAML 61 1 0 23 Feb 2025
Enhancing Adversarial Robustness of Vision-Language Models through Low-Rank Adaptation Yuheng Ji Yue Liu Zhicheng Zhang Zhao Zhang Yuting Zhao Gang Zhou Xingwei Zhang Xinwang Liu Xiaolong Zheng VLM 113 4 0 21 Feb 2025
Understanding and Rectifying Safety Perception Distortion in VLMs Xiaohan Zou Jian Kang George Kesidis Lu Lin 178 1 0 18 Feb 2025
Distraction is All You Need for Multimodal Large Language Model Jailbreaking Zuopeng Yang Jiluan Fan Anli Yan Erdun Gao Xin Lin Tao Li Kanghua mo Changyu Dong AAML 77 0 0 15 Feb 2025
Effective Black-Box Multi-Faceted Attacks Breach Vision Large Language Model Guardrails Yijun Yang L. Wang Xiao Yang Lanqing Hong Jun Zhu AAML 61 0 0 09 Feb 2025
Robust-LLaVA: On the Effectiveness of Large-Scale Robust Image Encoders for Multi-modal Large Language Models H. Malik Fahad Shamshad Muzammal Naseer Karthik Nandakumar F. Khan Salman Khan AAML MLLM VLM 68 0 0 03 Feb 2025
A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics Kai He Rui Mao Qika Lin Yucheng Ruan Xiang Lan Mengling Feng Erik Cambria LM&MA AILaw 93 154 0 28 Jan 2025
Playing Devil's Advocate: Unmasking Toxicity and Vulnerabilities in Large Vision-Language Models Abdulkadir Erol Trilok Padhi Agnik Saha Ugur Kursuncu Mehmet Emin Aktas 47 1 0 17 Jan 2025
Image-based Multimodal Models as Intruders: Transferable Multimodal Attacks on Video-based MLLMs Linhao Huang Xue Jiang Zhiqiang Wang Wentao Mo Xi Xiao Bo Han Yongjie Yin Feng Zheng AAML 51 2 0 02 Jan 2025
Adversarial Hubness in Multi-Modal Retrieval Tingwei Zhang Fnu Suya Rishi Jha Collin Zhang Vitaly Shmatikov AAML 83 1 0 18 Dec 2024
AdvDreamer Unveils: Are Vision-Language Models Truly Ready for Real-World 3D Variations? Shouwei Ruan Hanqin Liu Yao Huang Xiaoqi Wang Caixin Kang Hang Su Yinpeng Dong Xingxing Wei VGen 93 0 0 04 Dec 2024
Hijacking Vision-and-Language Navigation Agents with Adversarial Environmental Attacks Zijiao Yang Xiangxi Shi Eric Slyman Stefan Lee AAML 76 0 0 03 Dec 2024
Exploring Visual Vulnerabilities via Multi-Loss Adversarial Search for Jailbreaking Vision-Language Models Shuyang Hao Bryan Hooi Jiaheng Liu Kai-Wei Chang Zi Huang Yujun Cai AAML 92 1 0 27 Nov 2024
NEMO: Can Multimodal LLMs Identify Attribute-Modified Objects? Jiaxuan Li Junwen Mo MinhDuc Vo Akihiro Sugimoto Hideki Nakayama 85 0 0 26 Nov 2024
Chain of Attack: On the Robustness of Vision-Language Models Against Transfer-Based Adversarial Attacks Peng Xie Yequan Bie Jianda Mao Yangqiu Song Yang Wang Hao Chen Kani Chen AAML 69 1 0 24 Nov 2024
TAPT: Test-Time Adversarial Prompt Tuning for Robust Inference in Vision-Language Models Xin Wang Kai-xiang Chen Jiaming Zhang Jingjing Chen Xingjun Ma AAML VPVLM VLM 85 1 0 20 Nov 2024
PSA-VLM: Enhancing Vision-Language Model Safety through Progressive Concept-Bottleneck-Driven Alignment Zhendong Liu Yuanbi Nie Yingshui Tan Xiangyu Yue Qiushi Cui Chongjun Wang Xiaoyong Zhu Jian Xu Bo Zheng 73 0 0 18 Nov 2024
SoK: Unifying Cybersecurity and Cybersafety of Multimodal Foundation Models with an Information Theory Approach Ruoxi Sun Jiamin Chang Hammond Pearce Chaowei Xiao B. Li Qi Wu Surya Nepal Minhui Xue 35 0 0 17 Nov 2024
Exploring Response Uncertainty in MLLMs: An Empirical Evaluation under Misleading Scenarios Yunkai Dang Mengxi Gao Yibo Yan Xin Zou Yanggan Gu Aiwei Liu Xuming Hu 44 4 0 05 Nov 2024