Jailbreaking GPT-4V via Self-Adversarial Attacks with System Prompts

15 November 2023

Xiang Li

Lichao Sun

Papers citing "Jailbreaking GPT-4V via Self-Adversarial Attacks with System Prompts"

47 / 47 papers shown

Title
Attack and defense techniques in large language models: A survey and new perspectives Zhiyu Liao Kang Chen Yuanguo Lin Kangkang Li Yunxuan Liu Hefeng Chen Xingwang Huang Yuanhui Yu AAML 56 0 0 02 May 2025
DoomArena: A framework for Testing AI Agents Against Evolving Security Threats Léo Boisvert Mihir Bansal Chandra Kiran Reddy Evuru Gabriel Huang Abhay Puri ... Quentin Cappart Jason Stanley Alexandre Lacoste Alexandre Drouin Krishnamurthy Dvijotham 35 0 0 18 Apr 2025
A Domain-Based Taxonomy of Jailbreak Vulnerabilities in Large Language Models Carlos Peláez-González Andrés Herrera-Poyatos Cristina Zuheros David Herrera-Poyatos Virilo Tejedor F. Herrera AAML 24 0 0 07 Apr 2025
StyleRec: A Benchmark Dataset for Prompt Recovery in Writing Style Transformation Shenyang Liu Yang Gao Shaoyan Zhai Liqiang Wang 32 0 0 06 Apr 2025
Survey of Adversarial Robustness in Multimodal Large Language Models Chengze Jiang Zhuangzhuang Wang Minjing Dong Jie Gui AAML 63 0 0 18 Mar 2025
Tit-for-Tat: Safeguarding Large Vision-Language Models Against Jailbreak Attacks via Adversarial Defense Shuyang Hao Yijiao Wang Bryan Hooi Ming Yang Jiaheng Liu Chengcheng Tang Zi Huang Yujun Cai AAML 54 0 0 14 Mar 2025
Making Every Step Effective: Jailbreaking Large Vision-Language Models Through Hierarchical KV Equalization Shuyang Hao Yiwei Wang Bryan Hooi Jiaheng Liu Muhao Chen Zi Huang Yujun Cai AAML VLM 67 0 0 14 Mar 2025
Robust Concept Erasure Using Task Vectors Minh Pham Kelly O. Marshall Chinmay Hegde Niv Cohen 123 18 0 21 Feb 2025
Distraction is All You Need for Multimodal Large Language Model Jailbreaking Zuopeng Yang Jiluan Fan Anli Yan Erdun Gao Xin Lin Tao Li Kanghua mo Changyu Dong AAML 77 1 0 15 Feb 2025
Robust-LLaVA: On the Effectiveness of Large-Scale Robust Image Encoders for Multi-modal Large Language Models H. Malik Fahad Shamshad Muzammal Naseer Karthik Nandakumar Fahad Shahbaz Khan Salman Khan AAML MLLM VLM 68 0 0 03 Feb 2025
Can Safety Fine-Tuning Be More Principled? Lessons Learned from Cybersecurity David Williams-King Linh Le Adam Oberman Yoshua Bengio AAML 56 0 0 19 Jan 2025
Layer-Level Self-Exposure and Patch: Affirmative Token Mitigation for Jailbreak Attack Defense Yang Ouyang Hengrui Gu Shuhang Lin Wenyue Hua Jie Peng B. Kailkhura Tianlong Chen Kaixiong Zhou Kaixiong Zhou AAML 31 1 0 05 Jan 2025
Human-Readable Adversarial Prompts: An Investigation into LLM Vulnerabilities Using Situational Context Nilanjana Das Edward Raff Manas Gaur AAML 106 1 0 20 Dec 2024
Exploring Visual Vulnerabilities via Multi-Loss Adversarial Search for Jailbreaking Vision-Language Models Shuyang Hao Bryan Hooi Jiaheng Liu Kai-Wei Chang Zi Huang Yujun Cai AAML 92 1 0 27 Nov 2024
Jailbreak Attacks and Defenses against Multimodal Generative Models: A Survey Xuannan Liu Xing Cui Peipei Li Zekun Li Huaibo Huang Shuhan Xia Miaoxuan Zhang Yueying Zou Ran He AAML 67 8 0 14 Nov 2024
Recent advancements in LLM Red-Teaming: Techniques, Defenses, and Ethical Considerations Tarun Raheja Nilay Pochhi AAML 51 1 0 09 Oct 2024
AnyAttack: Towards Large-scale Self-supervised Adversarial Attacks on Vision-language Models Jiaming Zhang Junhong Ye Xingjun Ma Yige Li Yunfan Yang Jitao Sang Dit-Yan Yeung Dit-Yan Yeung AAML VLM 36 0 0 07 Oct 2024
Characterizing and Evaluating the Reliability of LLMs against Jailbreak Attacks Kexin Chen Yi Liu Donghai Hong Jiaying Chen Wenhai Wang 44 1 0 18 Aug 2024
Machine Unlearning in Generative AI: A Survey Zheyuan Liu Guangyao Dou Zhaoxuan Tan Yijun Tian Meng Jiang MU 31 14 0 30 Jul 2024
Can Large Language Models Automatically Jailbreak GPT-4V? Yuanwei Wu Yue Huang Yixin Liu Xiang Li Pan Zhou Lichao Sun SILM 40 1 0 23 Jul 2024
Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs) Apurv Verma Satyapriya Krishna Sebastian Gehrmann Madhavan Seshadri Anu Pradhan Tom Ault Leslie Barrett David Rabinowitz John Doucette Nhathai Phan 57 10 0 20 Jul 2024
Human-Interpretable Adversarial Prompt Attack on Large Language Models with Situational Context Nilanjana Das Edward Raff Manas Gaur AAML 35 2 0 19 Jul 2024
The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspective Zhen Qin Daoyuan Chen Wenhao Zhang Liuyi Yao Yilun Huang Bolin Ding Yaliang Li Shuiguang Deng 60 5 0 11 Jul 2024
A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends Daizong Liu Mingyu Yang Xiaoye Qu Pan Zhou Yu Cheng Wei Hu ELM AAML 30 25 0 10 Jul 2024
Threat Modelling and Risk Analysis for Large Language Model (LLM)-Powered Applications Stephen Burabari Tete 42 7 0 16 Jun 2024
JailbreakEval: An Integrated Toolkit for Evaluating Jailbreak Attempts Against Large Language Models Delong Ran Jinyuan Liu Yichen Gong Jingyi Zheng Xinlei He Tianshuo Cong Anyu Wang ELM 47 10 0 13 Jun 2024
MLLMGuard: A Multi-dimensional Safety Evaluation Suite for Multimodal Large Language Models Tianle Gu Zeyang Zhou Kexin Huang Dandan Liang Yixu Wang ... Keqing Wang Yujiu Yang Yan Teng Yu Qiao Yingchun Wang ELM 50 13 0 11 Jun 2024
SelfDefend: LLMs Can Defend Themselves against Jailbreaking in a Practical Manner Xunguang Wang Daoyuan Wu Zhenlan Ji Zongjie Li Pingchuan Ma Shuai Wang Yingjiu Li Yang Liu Ning Liu Juergen Rahmel AAML 79 8 0 08 Jun 2024
Ranking Manipulation for Conversational Search Engines Samuel Pfrommer Yatong Bai Tanmay Gautam Somayeh Sojoudi SILM 47 4 0 05 Jun 2024
DORY: Deliberative Prompt Recovery for LLM Lirong Gao Ru Peng Yiming Zhang Junbo Zhao 39 3 0 31 May 2024
Energy-Latency Manipulation of Multi-modal Large Language Models via Verbose Samples Kuofeng Gao Jindong Gu Yang Bai Shu-Tao Xia Philip Torr Wei Liu Zhifeng Li 71 11 0 25 Apr 2024
Advances and Open Challenges in Federated Learning with Foundation Models Chao Ren Han Yu Hongyi Peng Xiaoli Tang Anran Li ... A. Tan Bo Zhao Xiaoxiao Li Zengxiang Li Qiang Yang FedML AIFin AI4CE 78 7 0 23 Apr 2024
Unbridled Icarus: A Survey of the Potential Perils of Image Inputs in Multimodal Large Language Model Security Yihe Fan Yuxin Cao Ziyu Zhao Ziyao Liu Shaofeng Li 30 12 0 08 Apr 2024
Leveraging YOLO-World and GPT-4V LMMs for Zero-Shot Person Detection and Action Recognition in Drone Imagery Christian Limberg Artur Gonçalves Bastien Rigault Helmut Prendinger 40 5 0 02 Apr 2024
Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models Yifan Li Hangyu Guo Kun Zhou Wayne Xin Zhao Ji-Rong Wen 61 40 0 14 Mar 2024
Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformation Yunhao Gou Kai Chen Zhili Liu Lanqing Hong Hang Xu Zhenguo Li Dit-Yan Yeung James T. Kwok Yu Zhang MLLM 46 40 0 14 Mar 2024
LLMs Can Defend Themselves Against Jailbreaking in a Practical Manner: A Vision Paper Daoyuan Wu Shuaibao Wang Yang Liu Ning Liu AAML 39 7 0 24 Feb 2024
A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Models Zihao Xu Yi Liu Gelei Deng Yuekang Li S. Picek PILM AAML 41 35 0 21 Feb 2024
Safety of Multimodal Large Language Models on Images and Texts Xin Liu Yichen Zhu Yunshi Lan Chao Yang Yu Qiao 29 28 0 01 Feb 2024
Red-Teaming for Generative AI: Silver Bullet or Security Theater? Michael Feffer Anusha Sinha Wesley Hanwen Deng Zachary Chase Lipton Hoda Heidari AAML 38 67 0 29 Jan 2024
Fortifying Ethical Boundaries in AI: Advanced Strategies for Enhancing Security in Large Language Models Yunhong He Jianling Qiu Wei Zhang Zhe Yuan 32 3 0 27 Jan 2024
Tree of Attacks: Jailbreaking Black-Box LLMs Automatically Anay Mehrotra Manolis Zampetakis Paul Kassianik Blaine Nelson Hyrum Anderson Yaron Singer Amin Karbasi 30 204 0 04 Dec 2023
MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models Xin Liu Yichen Zhu Jindong Gu Yunshi Lan Chao Yang Yu Qiao 30 84 0 29 Nov 2023
GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts Jiahao Yu Xingwei Lin Zheng Yu Xinyu Xing SILM 117 303 0 19 Sep 2023
mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality Qinghao Ye Haiyang Xu Guohai Xu Jiabo Ye Ming Yan ... Junfeng Tian Qiang Qi Ji Zhang Feiyan Huang Jingren Zhou VLM MLLM 208 905 0 27 Apr 2023
The Parrot Dilemma: Human-Labeled vs. LLM-augmented Data in Classification Tasks Anders Giovanni Møller Jacob Aarup Dalsgaard Arianna Pera L. Aiello 81 35 0 26 Apr 2023
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 339 12,003 0 04 Mar 2022