A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily

14 November 2023

Papers citing "A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily"

50 / 81 papers shown

Title
Why Not Act on What You Know? Unleashing Safety Potential of LLMs via Self-Aware Guard Enhancement Peng Ding Jun Kuang Zongyu Wang Xuezhi Cao Xunliang Cai Jiajun Chen Shujian Huang 0 0 0 17 May 2025
PIG: Privacy Jailbreak Attack on LLMs via Gradient-based Iterative In-Context Optimization Yidan Wang Yanan Cao Yubing Ren Fang Fang Zheng-Shen Lin Binxing Fang PILM 44 0 0 15 May 2025
One Trigger Token Is Enough: A Defense Strategy for Balancing Safety and Usability in Large Language Models Haoran Gu Handing Wang Yi Mei Mengjie Zhang Yaochu Jin 27 0 0 12 May 2025
Red Teaming the Mind of the Machine: A Systematic Evaluation of Prompt Injection and Jailbreak Vulnerabilities in LLMs Chetan Pathade AAML SILM 59 0 0 07 May 2025
Cannot See the Forest for the Trees: Invoking Heuristics and Biases to Elicit Irrational Choices of LLMs Haoming Yang Ke Ma Xiaojun Jia Yingfei Sun Qianqian Xu Q. Huang AAML 159 0 0 03 May 2025
Prefill-Based Jailbreak: A Novel Approach of Bypassing LLM Safety Boundary Yakai Li Jiekang Hu Weiduan Sang Luping Ma Jing Xie Weijuan Zhang Aimin Yu Shijie Zhao Qingjia Huang Qihang Zhou AAML 52 0 0 28 Apr 2025
DETAM: Defending LLMs Against Jailbreak Attacks via Targeted Attention Modification Yu Li Han Jiang Zhihua Wei AAML 41 0 0 18 Apr 2025
AdaSteer: Your Aligned LLM is Inherently an Adaptive Jailbreak Defender Weixiang Zhao Jiahe Guo Yulin Hu Yang Deng An Zhang ... Xinyang Han Yanyan Zhao Bing Qin Tat-Seng Chua Ting Liu AAML LLMSV 43 0 0 13 Apr 2025
Geneshift: Impact of different scenario shift on Jailbreaking LLM Tianyi Wu Zhiwei Xue Yue Liu Jiaheng Zhang Bryan Hooi See-Kiong Ng 41 0 0 10 Apr 2025
Sugar-Coated Poison: Benign Generation Unlocks LLM Jailbreaking Yu-Hang Wu Yu-Jie Xiong Jie-Zhang AAML 30 0 0 08 Apr 2025
A Domain-Based Taxonomy of Jailbreak Vulnerabilities in Large Language Models Carlos Peláez-González Andrés Herrera-Poyatos Cristina Zuheros David Herrera-Poyatos Virilo Tejedor F. Herrera AAML 24 0 0 07 Apr 2025
Evolving Security in LLMs: A Study of Jailbreak Attacks and Defenses Zhengchun Shang Wenlan Wei AAML 45 1 0 02 Apr 2025
Prompt, Divide, and Conquer: Bypassing Large Language Model Safety Filters via Segmented and Distributed Prompt Processing Johan Wahréus Ahmed Mohamed Hussain P. Papadimitratos 58 0 0 27 Mar 2025
Dialogue Injection Attack: Jailbreaking LLMs through Context Manipulation Wenlong Meng Fan Zhang Wendao Yao Zhenyuan Guo Yongqian Li Chengkun Wei Wenzhi Chen AAML 40 1 0 11 Mar 2025
AgentSafe: Safeguarding Large Language Model-based Multi-agent Systems via Hierarchical Data Management Junyuan Mao Fanci Meng Yifan Duan Miao Yu Xiaojun Jia Fan Zhang Yuxuan Liang Kaidi Wang Qingsong Wen LLMAG AAML 53 1 0 06 Mar 2025
Foot-In-The-Door: A Multi-turn Jailbreak for LLMs Zixuan Weng Xiaolong Jin Jinyuan Jia Xiaotian Zhang AAML 149 0 0 27 Feb 2025
Beyond Surface-Level Patterns: An Essence-Driven Defense Framework Against Jailbreak Attacks in LLMs Shiyu Xiang Ansen Zhang Yanfei Cao Yang Fan Ronghao Chen AAML 62 0 0 26 Feb 2025
JailBench: A Comprehensive Chinese Security Assessment Benchmark for Large Language Models Shuyi Liu Simiao Cui Haoran Bu Yuming Shang Xi Zhang ELM 67 0 0 26 Feb 2025
Attention Eclipse: Manipulating Attention to Bypass LLM Safety-Alignment Pedram Zaree Md Abdullah Al Mamun Quazi Mishkatul Alam Yue Dong Ihsen Alouani Nael B. Abu-Ghazaleh AAML 41 0 0 24 Feb 2025
GuidedBench: Equipping Jailbreak Evaluation with Guidelines Ruixuan Huang Xunguang Wang Zongjie Li Daoyuan Wu Shuai Wang ALM ELM 61 0 0 24 Feb 2025
AISafetyLab: A Comprehensive Framework for AI Safety Evaluation and Improvement Zhexin Zhang Leqi Lei Junxiao Yang Xijie Huang Yida Lu ... Xianqi Lei C. Pan Lei Sha Hairu Wang Minlie Huang AAML 48 0 0 24 Feb 2025
SafeInt: Shielding Large Language Models from Jailbreak Attacks via Safety-Aware Representation Intervention Jiaqi Wu Chen Chen Chunyan Hou Xiaojie Yuan AAML 59 0 0 24 Feb 2025
A Mousetrap: Fooling Large Reasoning Models for Jailbreak with Chain of Iterative Chaos Yang Yao Xuan Tong Ruofan Wang Yixu Wang Lujundong Li Liang Liu Yan Teng Yishuo Wang LRM 51 3 0 19 Feb 2025
SafeEraser: Enhancing Safety in Multimodal Large Language Models through Multimodal Machine Unlearning Junkai Chen Zhijie Deng Kening Zheng Yibo Yan Shuliang Liu PeiJun Wu Peijie Jiang Jiaheng Liu Xuming Hu MU 69 3 0 18 Feb 2025
Jailbreaking to Jailbreak Jeremy Kritz Vaughn Robinson Robert Vacareanu Bijan Varjavand Michael Choi Bobby Gogov Scale Red Team Summer Yue Willow Primack Zifan Wang 209 1 0 09 Feb 2025
KDA: A Knowledge-Distilled Attacker for Generating Diverse Prompts to Jailbreak LLMs Buyun Liang Kwan Ho Ryan Chan D. Thaker Jinqi Luo René Vidal AAML 46 0 0 05 Feb 2025
When LLM Meets DRL: Advancing Jailbreaking Efficiency via DRL-guided Search Xuan Chen Yuzhou Nie Wenbo Guo Xiangyu Zhang 112 10 0 28 Jan 2025
HumorReject: Decoupling LLM Safety from Refusal Prefix via A Little Humor Zihui Wu Haichang Gao Jiacheng Luo Zhaoxiang Liu 41 0 0 23 Jan 2025
SATA: A Paradigm for LLM Jailbreak via Simple Assistive Task Linkage Xiaoning Dong Wenbo Hu Wei Xu Tianxing He 72 0 0 19 Dec 2024
Jailbreaking? One Step Is Enough! Weixiong Zheng Peijian Zeng Y. Li Hongyan Wu Nankai Lin Jianfei Chen Aimin Yang Yue Zhou AAML 81 0 0 17 Dec 2024
JailbreakLens: Interpreting Jailbreak Mechanism in the Lens of Representation and Circuit Zeqing He Zhibo Wang Zhixuan Chu Huiyu Xu Rui Zheng Kui Ren Chun Chen 57 3 0 17 Nov 2024
SQL Injection Jailbreak: A Structural Disaster of Large Language Models Jiawei Zhao Kejiang Chen Wenbo Zhang Nenghai Yu AAML 40 0 0 03 Nov 2024
Emoji Attack: Enhancing Jailbreak Attacks Against Judge LLM Detection Zhipeng Wei Yuqi Liu N. Benjamin Erichson AAML 53 1 0 01 Nov 2024
SG-Bench: Evaluating LLM Safety Generalization Across Diverse Tasks and Prompt Types Yutao Mou Shikun Zhang Wei Ye ELM 40 8 0 29 Oct 2024
Feint and Attack: Attention-Based Strategies for Jailbreaking and Protecting LLMs Rui Pu Chaozhuo Li Rui Ha Zejian Chen Litian Zhang Ziqiang Liu Lirong Qiu Xi Zhang AAML 34 1 0 18 Oct 2024
Toxic Subword Pruning for Dialogue Response Generation on Large Language Models Hongyuan Lu Wai Lam 17 0 0 05 Oct 2024
FlipAttack: Jailbreak LLMs via Flipping Yue Liu Xiaoxin He Miao Xiong Jinlan Fu Shumin Deng Bryan Hooi AAML 34 12 0 02 Oct 2024
Wait, but Tylenol is Acetaminophen... Investigating and Improving Language Models' Ability to Resist Requests for Misinformation Shan Chen Mingye Gao Kuleen Sasse Thomas Hartvigsen Brian Anthony Lizhou Fan Hugo J. W. L. Aerts Jack Gallifant Danielle S. Bitterman LM&MA 33 0 0 30 Sep 2024
PathSeeker: Exploring LLM Security Vulnerabilities with a Reinforcement Learning-Based Jailbreak Approach Zhihao Lin Wei Ma Mingyi Zhou Yanjie Zhao Haoyu Wang Yang Liu Jun Wang Li Li AAML 40 7 0 21 Sep 2024
Jailbreaking Large Language Models with Symbolic Mathematics Emet Bethany Mazal Bethany Juan Arturo Nolazco Flores S. Jha Peyman Najafirad AAML 18 3 0 17 Sep 2024
LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yet Nathaniel Li Ziwen Han Ian Steneker Willow Primack Riley Goodside Hugh Zhang Zifan Wang Cristina Menghini Summer Yue AAML MU 46 40 0 27 Aug 2024
Promoting Equality in Large Language Models: Identifying and Mitigating the Implicit Bias based on Bayesian Theory Yongxin Deng Xihe Qiu Xiaoyu Tan Jing Pan Chen Jue Zhijun Fang Yinghui Xu Wei Chu Yuan Qi 34 3 0 20 Aug 2024
Characterizing and Evaluating the Reliability of LLMs against Jailbreak Attacks Kexin Chen Yi Liu Donghai Hong Jiaying Chen Wenhai Wang 44 1 0 18 Aug 2024
Prefix Guidance: A Steering Wheel for Large Language Models to Defend Against Jailbreak Attacks Jiawei Zhao Kejiang Chen Xiaojian Yuan Weiming Zhang AAML 33 2 0 15 Aug 2024
WalledEval: A Comprehensive Safety Evaluation Toolkit for Large Language Models Prannaya Gupta Le Qi Yau Hao Han Low I-Shiang Lee Hugo Maximus Lim ... Jia Hng Koh Dar Win Liew Rishabh Bhardwaj Rajat Bhardwaj Soujanya Poria ELM LM&MA 60 4 0 07 Aug 2024
EnJa: Ensemble Jailbreak on Large Language Models Jiahao Zhang Zilong Wang Ruofan Wang Xingjun Ma Yu-Gang Jiang AAML 36 1 0 07 Aug 2024
Mission Impossible: A Statistical Perspective on Jailbreaking LLMs Jingtong Su Mingyu Lee SangKeun Lee 43 8 0 02 Aug 2024
Hallu-PI: Evaluating Hallucination in Multi-modal Large Language Models within Perturbed Inputs Peng Ding Jingyu Wu M. Girolami Dan Ma Xuezhi Cao Xunliang Cai Shi Chen T. J. Sullivan Shujian Huang AAML VLM MLLM 39 5 0 02 Aug 2024
The Dark Side of Function Calling: Pathways to Jailbreaking Large Language Models Zihui Wu Haichang Gao Jianping He Ping Wang 29 6 0 25 Jul 2024
LLMs can be Dangerous Reasoners: Analyzing-based Jailbreak Attack on Large Language Models Shi Lin Rongchang Li Xun Wang Changting Lin Xun Wang Wenpeng Xing Meng Han Meng Han 63 3 0 23 Jul 2024