v1v2 (latest)

Jailbreak Attacks and Defenses against Multimodal Generative Models: A Survey

14 November 2024

ArXiv (abs)PDF HTML Github (184★)

Papers citing "Jailbreak Attacks and Defenses against Multimodal Generative Models: A Survey"

50 / 60 papers shown

Title
Direct Unlearning Optimization for Robust and Safe Text-to-Image Models Yong-Hyun Park Sangdoo Yun Jin-Hwa Kim Junho Kim Geonhui Jang Yonghyun Jeong Junghyo Jo Gayoung Lee 144 19 0 17 Jan 2025
Open-Sora: Democratizing Efficient Video Production for All Zangwei Zheng Xiangyu Peng Tianji Yang Chenhui Shen Shenggui Li Hongxin Liu Yukun Zhou Tianyi Li Yang You VGen 163 256 0 31 Dec 2024
UniGuard: Towards Universal Safety Guardrails for Jailbreak Attacks on Multimodal Large Language Models Sejoon Oh Yiqiao Jin Megha Sharma Donghyun Kim Eric Ma Gaurav Verma Srijan Kumar 121 7 0 03 Nov 2024
BlueSuffix: Reinforced Blue Teaming for Vision-Language Models Against Jailbreak Attacks Yunhan Zhao Xiang Zheng Lin Luo Yige Li Xingjun Ma Yu-Gang Jiang VLM AAML 111 6 0 28 Oct 2024
SAFREE: Training-Free and Adaptive Guard for Safe Text-to-Image And Video Generation Jaehong Yoon Shoubin Yu Vaidehi Patil Huaxiu Yao Joey Tianyi Zhou 129 23 0 16 Oct 2024
Multimodal Pragmatic Jailbreak on Text-to-image Models Tong Liu Zhixin Lai Jiawen Wang Gengyuan Zhang Shuo Chen Philip Torr Vera Demberg Volker Tresp Jindong Gu 60 5 0 27 Sep 2024
Score Forgetting Distillation: A Swift, Data-Free Method for Machine Unlearning in Diffusion Models Tianqi Chen Shujian Zhang Mingyuan Zhou DiffM 160 5 0 17 Sep 2024
Securing Vision-Language Models with a Robust Encoder Against Jailbreak and Adversarial Attacks Md Zarif Hossain Ahmed Imteaj AAML VLM 78 6 0 11 Sep 2024
Perception-guided Jailbreak against Text-to-Image Models Yihao Huang Le Liang Tianlin Li Xiaojun Jia Run Wang Weikai Miao G. Pu Yang Liu 94 11 0 20 Aug 2024
DiffZOO: A Purely Query-Based Black-Box Attack for Red-teaming Text-to-Image Generative Model via Zeroth Order Optimization Pucheng Dang Xing Hu Dong Li Rui Zhang Qi Guo Kaidi Xu DiffM 89 7 0 18 Aug 2024
BaThe: Defense against the Jailbreak Attack in Multimodal Large Language Models by Treating Harmful Instruction as Backdoor Trigger Yulin Chen Haoran Li Zihao Zheng Zihao Zheng Yangqiu Song Bryan Hooi 139 7 0 17 Aug 2024
CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer Zhuoyi Yang Jiayan Teng Wendi Zheng Ming Ding Shiyu Huang ... Weihan Wang Yean Cheng Xiaotao Gu Yuxiao Dong Jie Tang DiffM VGen 243 565 0 12 Aug 2024
Attacks and Defenses for Generative Diffusion Models: A Comprehensive Survey V. T. Truong Luan Ba Dang Long Bao Le DiffM MedIm 104 18 0 06 Aug 2024
Fuzz-Testing Meets LLM-Based Agents: An Automated and Efficient Framework for Jailbreaking Text-To-Image Generation Models Yingkai Dong Xiangtao Meng Ning Yu Zheng Li Shanqing Guo LLMAG 92 17 0 01 Aug 2024
T2VSafetyBench: Evaluating the Safety of Text-to-Video Generative Models Yibo Miao Yifan Zhu Yinpeng Dong Lijia Yu Jun Zhu Xiao-Shan Gao EGVM 112 20 0 08 Jul 2024
MMFakeBench: A Mixed-Source Multimodal Misinformation Detection Benchmark for LVLMs Xuannan Liu Zekun Li Peipei Li Shuhan Xia Xing Cui Linzhi Huang Huaibo Huang Weihong Deng Zhaofeng He 133 23 0 13 Jun 2024
Pruning for Robust Concept Erasing in Diffusion Models Tianyun Yang Juan Cao Chang Xu 95 14 0 26 May 2024
Unlearning Concepts in Diffusion Model via Concept Domain Correction and Concept Preserving Gradient Yongliang Wu Shiji Zhou Mingzhuo Yang Lianzhe Wang Wenbo Zhu Heng Chang Xiao Zhou Xu Yang Xu Yang 136 21 0 24 May 2024
UPAM: Unified Prompt Attack in Text-to-Image Generation Models Against Both Textual Filters and Visual Checkers Duo Peng Qi Ke Jun Liu 81 4 0 18 May 2024
Chameleon: Mixed-Modal Early-Fusion Foundation Models Chameleon Team MLLM 201 338 0 16 May 2024
Latent Guard: a Safety Framework for Text-to-image Generation Runtao Liu Ashkan Khakzar Jindong Gu Qifeng Chen Philip Torr Fabio Pizzati 87 31 0 11 Apr 2024
Jailbreaking Prompt Attack: A Controllable Adversarial Attack against Diffusion Models Jiachen Ma Anda Cao Zhiqing Xiao Jie Zhang Chaonan Ye Chao Ye Junbo Zhao 114 33 0 02 Apr 2024
Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models Yifan Li Hangyu Guo Kun Zhou Wayne Xin Zhao Ji-Rong Wen 119 56 0 14 Mar 2024
AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Prompting Yu Wang Xiaogeng Liu Yu-Feng Li Muhao Chen Chaowei Xiao AAML 85 61 0 14 Mar 2024
Sora as an AGI World Model? A Complete Survey on Text-to-Video Generation Joseph Cho Fachrina Dewi Puspitasari Sheng Zheng Jingyao Zheng Lik-Hang Lee Tae-Ho Kim Choong Seon Hong Chaoning Zhang EGVM VGen 93 42 0 08 Mar 2024
ImgTrojan: Jailbreaking Vision-Language Models with ONE Image Xijia Tao Shuai Zhong Lei Li Qi Liu Lingpeng Kong 122 30 0 05 Mar 2024
GuardT2I: Defending Text-to-Image Models from Adversarial Prompts Yijun Yang Ruiyuan Gao Xiao Yang Jianyuan Zhong Qiang Xu 82 20 0 03 Mar 2024
Diffusion Model-Based Image Editing: A Survey Yi Huang Jiancheng Huang Yifan Liu Mingfu Yan Jiaxi Lv Jianzhuang Liu Wei Xiong He Zhang Liangliang Cao Liangliang Cao EGVM 220 103 0 27 Feb 2024
Universal Prompt Optimizer for Safe Text-to-Image Generation Zongyu Wu Hongcheng Gao Yueze Wang Xiang Zhang Suhang Wang EGVM 48 11 0 16 Feb 2024
InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidance Pengyu Wang Dong Zhang Linyang Li Chenkun Tan Xinghao Wang Ke Ren Botian Jiang Xipeng Qiu LLMSV 91 49 0 20 Jan 2024
MLLM-Protector: Ensuring MLLM's Safety without Hurting Performance Renjie Pi Tianyang Han Jianshu Zhang Yueqi Xie Boyao Wang Qing Lian Hanze Dong Jipeng Zhang Tong Zhang AAML 101 69 0 05 Jan 2024
Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations Hakan Inan Kartikeya Upasani Jianfeng Chi Rashi Rungta Krithika Iyer ... Michael Tontchev Qing Hu Brian Fuller Davide Testuggine Madian Khabsa AI4MH 165 463 0 07 Dec 2023
FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts Yichen Gong Delong Ran Jinyuan Liu Conglei Wang Tianshuo Cong Anyu Wang Sisi Duan Xiaoyun Wang MLLM 224 160 0 09 Nov 2023
To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Now Yimeng Zhang Jinghan Jia Xin Chen Aochuan Chen Yihua Zhang Jiancheng Liu Ke Ding Sijia Liu DiffM 147 100 0 18 Oct 2023
Ring-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models? Yu-Lin Tsai Chia-Yi Hsu Chulin Xie Chih-Hsun Lin Jia-You Chen Yue Liu Pin-Yu Chen Chia-Mu Yu Chun-ying Huang DiffM 100 90 0 16 Oct 2023
Implicit Concept Removal of Diffusion Models Zhili Liu Kai Chen Yifan Zhang Jianhua Han Lanqing Hong Hang Xu Zhenguo Li Dit-Yan Yeung James T. Kwok 61 14 0 09 Oct 2023
SurrogatePrompt: Bypassing the Safety Filter of Text-To-Image Models via Substitution Zhongjie Ba Jieming Zhong Jiachen Lei Pengyu Cheng Qinglong Wang Zhan Qin Peng Kuang Kui Ren 70 22 0 25 Sep 2023
GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts Jiahao Yu Xingwei Lin Zheng Yu Xinyu Xing SILM 219 352 0 19 Sep 2023
Can I Trust Your Answer? Visually Grounded Video Question Answering Junbin Xiao Angela Yao Yicong Li Tat-Seng Chua 120 61 0 04 Sep 2023
Image Hijacks: Adversarial Images can Control Generative Models at Runtime Luke Bailey Euan Ong Stuart J. Russell Scott Emmons VLM MLLM 89 87 0 01 Sep 2023
Universal and Transferable Adversarial Attacks on Aligned Language Models Andy Zou Zifan Wang Nicholas Carlini Milad Nasr J. Zico Kolter Matt Fredrikson 295 1,518 0 27 Jul 2023
Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models Erfan Shayegani Yue Dong Nael B. Abu-Ghazaleh 109 152 0 26 Jul 2023
Visual Adversarial Examples Jailbreak Aligned Large Language Models Xiangyu Qi Kaixuan Huang Ashwinee Panda Peter Henderson Mengdi Wang Prateek Mittal AAML 100 171 0 22 Jun 2023
SneakyPrompt: Jailbreaking Text-to-image Generative Models Yuchen Yang Bo Hui Haolin Yuan Neil Zhenqiang Gong Yinzhi Cao EGVM 155 92 0 20 May 2023
InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning Wenliang Dai Junnan Li Dongxu Li A. M. H. Tiong Junqi Zhao Weisheng Wang Boyang Albert Li Pascale Fung Steven C. H. Hoi MLLM VLM 150 2,098 0 11 May 2023
Visual Instruction Tuning Haotian Liu Chunyuan Li Qingyang Wu Yong Jae Lee SyDa VLM MLLM 571 4,925 0 17 Apr 2023
Composer: Creative and Controllable Image Synthesis with Composable Conditions Lianghua Huang Di Chen Yu Liu Yujun Shen Deli Zhao Jingren Zhou DiffM 83 290 0 20 Feb 2023
T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models Chong Mou Xintao Wang Liangbin Xie Yanze Wu Shuai Liu Zhongang Qi Ying Shan Xiaohu Qie DiffM 138 1,034 0 16 Feb 2023
Adding Conditional Control to Text-to-Image Diffusion Models Lvmin Zhang Anyi Rao Maneesh Agrawala AI4CE 184 4,180 1 10 Feb 2023
Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models P. Schramowski Manuel Brack Bjorn Deiseroth Kristian Kersting 141 310 0 09 Nov 2022