On Evaluating Adversarial Robustness of Large Vision-Language Models

26 May 2023

Papers citing "On Evaluating Adversarial Robustness of Large Vision-Language Models"

50 / 136 papers shown

Title
Replace-then-Perturb: Targeted Adversarial Attacks With Visual Reasoning for Vision-Language Models Jonggyu Jang Hyeonsu Lyu Jungyeon Koh H. Yang VLM AAML 45 0 0 01 Nov 2024
Sensitivity of Generative VLMs to Semantically and Lexically Altered Prompts Sri Harsha Dumpala Aman Jaiswal Chandramouli Shama Sastry E. Milios Sageev Oore Hassan Sajjad VLM 26 2 0 16 Oct 2024
SAFREE: Training-Free and Adaptive Guard for Safe Text-to-Image And Video Generation Jaehong Yoon Shoubin Yu Vaidehi Patil Huaxiu Yao Joey Tianyi Zhou 79 15 0 16 Oct 2024
R-Bench: Are your Large Multimodal Model Robust to Real-world Corruptions? Chunyi Li J. Zhang Zicheng Zhang H. Wu Yuan Tian ... Guo Lu Xiaohong Liu Xiongkuo Min Weisi Lin Guangtao Zhai AAML 39 3 0 07 Oct 2024
AnyAttack: Towards Large-scale Self-supervised Adversarial Attacks on Vision-language Models Jiaming Zhang Junhong Ye Xingjun Ma Yige Li Yunfan Yang Jitao Sang Dit-Yan Yeung Dit-Yan Yeung AAML VLM 34 0 0 07 Oct 2024
SCA: Highly Efficient Semantic-Consistent Unrestricted Adversarial Attack Zihao Pan Weibin Wu Yuhang Cao Zibin Zheng DiffM AAML 65 1 0 03 Oct 2024
Toward a Holistic Evaluation of Robustness in CLIP Models Weijie Tu Weijian Deng Tom Gedeon VLM 38 5 0 02 Oct 2024
MFCLIP: Multi-modal Fine-grained CLIP for Generalizable Diffusion Face Forgery Detection Yaning Zhang Tianyi Wang Zitong Yu Zan Gao Linlin Shen Shengyong Chen DiffM 65 3 0 15 Sep 2024
Securing Vision-Language Models with a Robust Encoder Against Jailbreak and Adversarial Attacks Md Zarif Hossain Ahmed Imteaj AAML VLM 43 3 0 11 Sep 2024
PromptSmooth: Certifying Robustness of Medical Vision-Language Models via Prompt Learning Noor Hussein Fahad Shamshad Muzammal Naseer Karthik Nandakumar AAML 54 3 0 29 Aug 2024
Ask, Attend, Attack: A Effective Decision-Based Black-Box Targeted Attack for Image-to-Text Models Qingyuan Zeng Zhenzhong Wang Yiu-ming Cheung Min Jiang AAML 42 1 0 16 Aug 2024
AdvQDet: Detecting Query-Based Adversarial Attacks with Adversarial Contrastive Prompt Tuning Xin Wang Kai-xiang Chen Xingjun Ma Zhineng Chen Jingjing Chen Yu-Gang Jiang AAML 43 3 0 04 Aug 2024
Joint Universal Adversarial Perturbations with Interpretations Liang-bo Ning Zeyu Dai Wenqi Fan Jingran Su Chao Pan Luning Wang Qing Li AAML 45 2 0 03 Aug 2024
Assessing Brittleness of Image-Text Retrieval Benchmarks from Vision-Language Models Perspective Mariya Hendriksen Shuo Zhang R. Reinanda Mohamed Yahya Edgar Meij Maarten de Rijke 54 0 0 21 Jul 2024
Towards Adversarially Robust Vision-Language Models: Insights from Design Choices and Prompt Formatting Techniques Rishika Bhagwatkar Shravan Nayak Reza Bayat Alexis Roger Daniel Z Kaplan P. Bashivan Irina Rish AAML VLM 39 1 0 15 Jul 2024
Refusing Safe Prompts for Multi-modal Large Language Models Zedian Shao Hongbin Liu Yuepeng Hu Neil Zhenqiang Gong MLLM LRM 41 1 0 12 Jul 2024
A Comprehensive Survey on the Security of Smart Grid: Challenges, Mitigations, and Future Research Opportunities Arastoo Zibaeirad Farnoosh Koleini Shengping Bi Tao Hou Tao Wang AAML 44 14 0 10 Jul 2024
A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends Daizong Liu Mingyu Yang Xiaoye Qu Pan Zhou Yu Cheng Wei Hu ELM AAML 30 25 0 10 Jul 2024
ElecBench: a Power Dispatch Evaluation Benchmark for Large Language Models Xiyuan Zhou Huan Zhao Yuheng Cheng Yuji Cao Gaoqi Liang Guolong Liu Wenxuan Liu Yan Xu Junhua Zhao ELM 38 6 0 07 Jul 2024
Video Watermarking: Safeguarding Your Video from (Unauthorized) Annotations by Video-based LLMs Jinmin Li Kuofeng Gao Yang Bai Jingyun Zhang Shu-Tao Xia 48 4 0 02 Jul 2024
Image-to-Text Logic Jailbreak: Your Imagination can Help You Do Anything Xiaotian Zou Ke Li Yongkang Chen MLLM 42 2 0 01 Jul 2024
JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Models Haibo Jin Leyang Hu Xinuo Li Peiyan Zhang Chonghan Chen Jun Zhuang Haohan Wang PILM 36 26 0 26 Jun 2024
Automatic benchmarking of large multimodal models via iterative experiment programming Alessandro Conti Enrico Fini Paolo Rota Yiming Wang Massimiliano Mancini Elisa Ricci 46 0 0 18 Jun 2024
Harmonizing Feature Maps: A Graph Convolutional Approach for Enhancing Adversarial Robustness Kejia Zhang Juanjuan Weng Junwei Wu Guoqing Yang Shaozi Li Zhiming Luo AAML 49 1 0 17 Jun 2024
SPA-VL: A Comprehensive Safety Preference Alignment Dataset for Vision Language Model Yongting Zhang Lu Chen Guodong Zheng Yifeng Gao Rui Zheng ... Yu Qiao Xuanjing Huang Feng Zhao Tao Gui Jing Shao VLM 85 24 0 17 Jun 2024
MirrorCheck: Efficient Adversarial Defense for Vision-Language Models Samar Fares Klea Ziu Toluwani Aremu N. Durasov Martin Takáč Pascal Fua Karthik Nandakumar Ivan Laptev VLM AAML 40 4 0 13 Jun 2024
SelfDefend: LLMs Can Defend Themselves against Jailbreaking in a Practical Manner Xunguang Wang Daoyuan Wu Zhenlan Ji Zongjie Li Pingchuan Ma Shuai Wang Yingjiu Li Yang Liu Ning Liu Juergen Rahmel AAML 76 8 0 08 Jun 2024
Jailbreak Vision Language Models via Bi-Modal Adversarial Prompt Zonghao Ying Aishan Liu Tianyuan Zhang Zhengmin Yu Siyuan Liang Xianglong Liu Dacheng Tao AAML 37 26 0 06 Jun 2024
Typography Leads Semantic Diversifying: Amplifying Adversarial Transferability across Multimodal Large Language Models Hao-Ran Cheng Erjia Xiao Jiahang Cao Le Yang Kaidi Xu Jindong Gu Renjing Xu AAML 64 7 0 30 May 2024
Efficient Black-box Adversarial Attacks via Bayesian Optimization Guided by a Function Prior Shuyu Cheng Yibo Miao Yinpeng Dong Xiao Yang Xiao-Shan Gao Jun Zhu AAML 35 3 0 29 May 2024
TIMA: Text-Image Mutual Awareness for Balancing Zero-Shot Adversarial Robustness and Generalization Ability Fengji Ma Li Liu Hei Victor Cheng VLM 33 0 0 27 May 2024
Safety Alignment for Vision Language Models Zhendong Liu Yuanbi Nie Yingshui Tan Xiangyu Yue Qiushi Cui Chongjun Wang Xiaoyong Zhu Bo Zheng VLM MLLM 98 7 0 22 May 2024
Adversarial Robustness for Visual Grounding of Multimodal Large Language Models Kuofeng Gao Yang Bai Jiawang Bai Yong Yang Shu-Tao Xia AAML 24 18 0 16 May 2024
Assessing Adversarial Robustness of Large Language Models: An Empirical Study Zeyu Yang Zhao Meng Xiaochen Zheng Roger Wattenhofer ELM AAML 31 7 0 04 May 2024
Revisiting the Adversarial Robustness of Vision Language Models: a Multimodal Perspective Wanqi Zhou Shuanghao Bai Qibin Zhao Badong Chen VLM AAML 44 5 0 30 Apr 2024
Energy-Latency Manipulation of Multi-modal Large Language Models via Verbose Samples Kuofeng Gao Jindong Gu Yang Bai Shu-Tao Xia Philip H. S. Torr Wei Liu Zhifeng Li 71 11 0 25 Apr 2024
BRAVE: Broadening the visual encoding of vision-language models Ouguzhan Fatih Kar A. Tonioni Petra Poklukar Achin Kulshrestha Amir Zamir Federico Tombari MLLM VLM 50 25 0 10 Apr 2024
Unbridled Icarus: A Survey of the Potential Perils of Image Inputs in Multimodal Large Language Model Security Yihe Fan Yuxin Cao Ziyu Zhao Ziyao Liu Shaofeng Li 30 12 0 08 Apr 2024
ImageNet-D: Benchmarking Neural Network Robustness on Diffusion Synthetic Object Chenshuang Zhang Fei Pan Junmo Kim In So Kweon Chengzhi Mao 51 9 1 27 Mar 2024
FMM-Attack: A Flow-based Multi-modal Adversarial Attack on Video-based LLMs Jinmin Li Kuofeng Gao Yang Bai Jingyun Zhang Shu-Tao Xia Yisen Wang AAML 32 7 0 20 Mar 2024
As Firm As Their Foundations: Can open-sourced foundation models be used to create adversarial examples for downstream tasks? Anjun Hu Jindong Gu Francesco Pinto Konstantinos Kamnitsas Philip H. S. Torr AAML SILM 37 5 0 19 Mar 2024
Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models Yifan Li Hangyu Guo Kun Zhou Wayne Xin Zhao Ji-Rong Wen 61 39 0 14 Mar 2024
An Image Is Worth 1000 Lies: Adversarial Transferability across Prompts on Vision-Language Models Haochen Luo Jindong Gu Fengyuan Liu Philip H. S. Torr VLM VPVLM AAML 55 20 0 14 Mar 2024
Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformation Yunhao Gou Kai Chen Zhili Liu Lanqing Hong Hang Xu Zhenguo Li Dit-Yan Yeung James T. Kwok Yu Zhang MLLM 46 37 0 14 Mar 2024
One Prompt Word is Enough to Boost Adversarial Robustness for Pre-trained Vision-Language Models Lin Li Haoyan Guan Jianing Qiu Michael W. Spratling AAML VLM VPVLM 31 21 0 04 Mar 2024
NPHardEval4V: A Dynamic Reasoning Benchmark of Multimodal Large Language Models Lizhou Fan Wenyue Hua Xiang Li Kaijie Zhu Mingyu Jin ... Haoyang Ling Jinkui Chi Jindong Wang Xin Ma Yongfeng Zhang LRM 40 14 0 04 Mar 2024
Stop Reasoning! When Multimodal LLMs with Chain-of-Thought Reasoning Meets Adversarial Images Zefeng Wang Zhen Han Shuo Chen Fan Xue Zifeng Ding Xun Xiao Volker Tresp Philip H. S. Torr Jindong Gu LRM 44 14 0 22 Feb 2024
Robust CLIP: Unsupervised Adversarial Fine-Tuning of Vision Embeddings for Robust Large Vision-Language Models Christian Schlarmann Naman D. Singh Francesco Croce Matthias Hein VLM AAML 47 38 0 19 Feb 2024
Test-Time Backdoor Attacks on Multimodal Large Language Models Dong Lu Tianyu Pang Chao Du Qian Liu Xianjun Yang Min-Bin Lin AAML 56 21 0 13 Feb 2024
Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast Xiangming Gu Xiaosen Zheng Tianyu Pang Chao Du Qian Liu Ye Wang Jing Jiang Min-Bin Lin LLMAG LM&Ro 37 48 0 13 Feb 2024