Unbridled Icarus: A Survey of the Potential Perils of Image Inputs in Multimodal Large Language Model Security

8 April 2024

Papers citing "Unbridled Icarus: A Survey of the Potential Perils of Image Inputs in Multimodal Large Language Model Security"

30 / 30 papers shown

Title
An Enhanced Large Language Model For Cross Modal Query Understanding System Using DL-KeyBERT Based CAZSSCL-MPGPT Shreya Singh 103 0 0 24 Feb 2025
Learn What You Want to Unlearn: Unlearning Inversion Attacks against Machine Unlearning Hongsheng Hu Shuo Wang Tian Dong Minhui Xue AAML 74 28 0 04 Apr 2024
Threats, Attacks, and Defenses in Machine Unlearning: A Survey Ziyao Liu Huanyi Ye Chen Chen Yongsen Zheng K. Lam AAML MU 105 32 0 20 Mar 2024
Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models Yifan Li Hangyu Guo Kun Zhou Wayne Xin Zhao Ji-Rong Wen 116 53 0 14 Mar 2024
Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformation Yunhao Gou Kai Chen Zhili Liu Lanqing Hong Hang Xu Zhenguo Li Dit-Yan Yeung James T. Kwok Yu Zhang MLLM 102 52 0 14 Mar 2024
AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Prompting Yu Wang Xiaogeng Liu Yu-Feng Li Muhao Chen Chaowei Xiao AAML 81 60 0 14 Mar 2024
ImgTrojan: Jailbreaking Vision-Language Models with ONE Image Xijia Tao Shuai Zhong Lei Li Qi Liu Lingpeng Kong 112 29 0 05 Mar 2024
Eight Methods to Evaluate Robust Unlearning in LLMs Aengus Lynch Phillip Guo Aidan Ewart Stephen Casper Dylan Hadfield-Menell ELM MU 98 81 0 26 Feb 2024
The Wolf Within: Covert Injection of Malice into MLLM Societies via an MLLM Operative Zhen Tan Chengshuai Zhao Raha Moraffah Yifan Li Yu Kong Tianlong Chen Huan Liu 70 17 0 20 Feb 2024
Test-Time Backdoor Attacks on Multimodal Large Language Models Dong Lu Tianyu Pang Chao Du Qian Liu Xianjun Yang Min Lin AAML 131 24 0 13 Feb 2024
Shadowcast: Stealthy Data Poisoning Attacks Against Vision-Language Models Yuancheng Xu Jiarui Yao Manli Shu Yanchao Sun Zichu Wu Ning Yu Tom Goldstein Furong Huang AAML 108 25 0 05 Feb 2024
Vision-LLMs Can Fool Themselves with Self-Generated Typographic Attacks Maan Qraitem Nazia Tasnim Piotr Teterwak Kate Saenko Bryan A. Plummer AAML VLM 78 12 0 01 Feb 2024
Red Teaming Visual Language Models Mukai Li Lei Li Yuwei Yin Masood Ahmed Zhenguang Liu Qi Liu VLM 99 38 0 23 Jan 2024
InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidance Pengyu Wang Dong Zhang Linyang Li Chenkun Tan Xinghao Wang Ke Ren Botian Jiang Xipeng Qiu LLMSV 82 49 0 20 Jan 2024
Towards Efficient and Certified Recovery from Poisoning Attacks in Federated Learning Yu Jiang Jiyuan Shen Ziyao Liu Chee Wei Tan Kwok-Yan Lam AAML FedML 90 5 0 16 Jan 2024
MLLM-Protector: Ensuring MLLM's Safety without Hurting Performance Renjie Pi Tianyang Han Jianshu Zhang Yueqi Xie Boyao Wang Qing Lian Hanze Dong Jipeng Zhang Tong Zhang AAML 79 69 0 05 Jan 2024
SA-Attack: Improving Adversarial Transferability of Vision-Language Pre-training Models via Self-Augmentation Bangyan He Xiaojun Jia Siyuan Liang Tianrui Lou Yang Liu Xiaochun Cao AAML VLM 89 29 0 08 Dec 2023
OT-Attack: Enhancing Adversarial Transferability of Vision-Language Models via Optimal Transport Optimization Dongchen Han Xiaojun Jia Yang Bai Jindong Gu Yang Liu Xiaochun Cao VLM 70 26 0 07 Dec 2023
DRESS: Instructing Large Vision-Language Models to Align and Interact with Humans via Natural Language Feedback Yangyi Chen Karan Sikka Michael Cogswell Heng Ji Ajay Divakaran 108 72 0 16 Nov 2023
FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts Yichen Gong Delong Ran Jinyuan Liu Conglei Wang Tianshuo Cong Anyu Wang Sisi Duan Xiaoyun Wang MLLM 220 158 0 09 Nov 2023
Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks Erfan Shayegani Md Abdullah Al Mamun Yu Fu Pedram Zaree Yue Dong Nael B. Abu-Ghazaleh AAML 225 163 0 16 Oct 2023
VLATTACK: Multimodal Adversarial Attacks on Vision-Language Tasks via Pre-trained Models Ziyi Yin Muchao Ye Tianrong Zhang Tianyu Du Jinguo Zhu Han Liu Jinghui Chen Ting Wang Fenglong Ma AAML VLM CoGe 89 44 0 07 Oct 2023
Image Hijacks: Adversarial Images can Control Generative Models at Runtime Luke Bailey Euan Ong Stuart J. Russell Scott Emmons VLM MLLM 73 87 0 01 Sep 2023
Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models Erfan Shayegani Yue Dong Nael B. Abu-Ghazaleh 103 152 0 26 Jul 2023
Visual Adversarial Examples Jailbreak Aligned Large Language Models Xiangyu Qi Kaixuan Huang Ashwinee Panda Peter Henderson Mengdi Wang Prateek Mittal AAML 95 170 0 22 Jun 2023
On Evaluating Adversarial Robustness of Large Vision-Language Models Yunqing Zhao Tianyu Pang Chao Du Xiao Yang Chongxuan Li Ngai-Man Cheung Min Lin VLM AAML MLLM 131 180 0 26 May 2023
A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation Xiaowei Huang Wenjie Ruan Wei Huang Gao Jin Yizhen Dong ... Sihao Wu Peipei Xu Dengyu Wu André Freitas Mustafa A. Mustafa ALM 113 93 0 19 May 2023
Robust Contrastive Language-Image Pre-training against Data Poisoning and Backdoor Attacks Wenhan Yang Jingdong Gao Baharan Mirzasoleiman VLM 165 20 0 13 Mar 2023
High-Resolution Image Synthesis with Latent Diffusion Models Robin Rombach A. Blattmann Dominik Lorenz Patrick Esser Bjorn Ommer 3DV 496 15,768 0 20 Dec 2021
Hidden Backdoors in Human-Centric Language Models Shaofeng Li Hui Liu Tian Dong Benjamin Zi Hao Zhao Minhui Xue Haojin Zhu Jialiang Lu SILM 100 154 0 01 May 2021