One Trigger Token Is Enough: A Defense Strategy for Balancing Safety and Usability in Large Language Models

12 May 2025

Papers citing "One Trigger Token Is Enough: A Defense Strategy for Balancing Safety and Usability in Large Language Models"

34 / 34 papers shown

Title
ParetoHqD: Fast Offline Multiobjective Alignment of Large Language Models using Pareto High-quality Data Haoran Gu Handing Wang Yi Mei Mengjie Zhang Yaochu Jin 70 1 0 23 Apr 2025
Align in Depth: Defending Jailbreak Attacks via Progressive Answer Detoxification Yingjie Zhang Tong Liu Zhe Zhao Guozhu Meng Kai Chen AAML 91 1 0 14 Mar 2025
Prefix Guidance: A Steering Wheel for Large Language Models to Defend Against Jailbreak Attacks Jiawei Zhao Kejiang Chen Xiaojian Yuan Weiming Zhang AAML 58 2 0 15 Aug 2024
Adversarial Contrastive Decoding: Boosting Safety Alignment of Large Language Models via Opposite Prompt Optimization Zhengyue Zhao Xiaoyun Zhang Kaidi Xu Xing Hu Rui Zhang Zidong Du Qi Guo Yunji Chen 62 8 0 24 Jun 2024
Safety Alignment Should Be Made More Than Just a Few Tokens Deep Xiangyu Qi Ashwinee Panda Kaifeng Lyu Xiao Ma Subhrajit Roy Ahmad Beirami Prateek Mittal Peter Henderson 98 137 0 10 Jun 2024
How Alignment and Jailbreak Work: Explain LLM Safety through Intermediate Hidden States Zhenhong Zhou Haiyang Yu Xinghua Zhang Rongwu Xu Fei Huang Yongbin Li 86 41 0 09 Jun 2024
SelfDefend: LLMs Can Defend Themselves against Jailbreaking in a Practical Manner Xunguang Wang Daoyuan Wu Zhenlan Ji Zongjie Li Pingchuan Ma Shuai Wang Yingjiu Li Yang Liu Ning Liu Juergen Rahmel AAML 150 14 0 08 Jun 2024
Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editing Wei Zhao Zhe Li Yige Li Ye Zhang Junfeng Sun KELM AAML 43 32 0 28 May 2024
Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks Maksym Andriushchenko Francesco Croce Nicolas Flammarion AAML 155 220 0 02 Apr 2024
Attacks, Defenses and Evaluations for LLM Conversation Safety: A Survey Zhichen Dong Zhanhui Zhou Chao Yang Jing Shao Yu Qiao ELM 106 67 0 14 Feb 2024
SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decoding Zhangchen Xu Fengqing Jiang Luyao Niu Jinyuan Jia Bill Yuchen Lin Radha Poovendran AAML 167 110 0 14 Feb 2024
How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs Yi Zeng Hongpeng Lin Jingwen Zhang Diyi Yang Ruoxi Jia Weiyan Shi 90 312 0 12 Jan 2024
Bypassing the Safety Training of Open-Source LLMs with Priming Attacks Jason Vega Isha Chaudhary Changming Xu Gagandeep Singh AAML 62 24 0 19 Dec 2023
Tree of Attacks: Jailbreaking Black-Box LLMs Automatically Anay Mehrotra Manolis Zampetakis Paul Kassianik Blaine Nelson Hyrum Anderson Yaron Singer Amin Karbasi 81 266 0 04 Dec 2023
The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning Bill Yuchen Lin Abhilasha Ravichander Ximing Lu Nouha Dziri Melanie Sclar Khyathi Chandu Chandra Bhagavatula Yejin Choi 59 196 0 04 Dec 2023
A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily Peng Ding Jun Kuang Dan Ma Xuezhi Cao Yunsen Xian Jiajun Chen Shujian Huang AAML 78 121 0 14 Nov 2023
DeepInception: Hypnotize Large Language Model to Be Jailbreaker Xuan Li Zhanke Zhou Jianing Zhu Jiangchao Yao Tongliang Liu Bo Han 91 189 0 06 Nov 2023
Jailbreaking Black Box Large Language Models in Twenty Queries Patrick Chao Alexander Robey Yan Sun Hamed Hassani George J. Pappas Eric Wong AAML 113 700 0 12 Oct 2023
Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations Zeming Wei Yifei Wang Ang Li Yichuan Mo Yisen Wang 98 277 0 10 Oct 2023
Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! Xiangyu Qi Yi Zeng Tinghao Xie Pin-Yu Chen Ruoxi Jia Prateek Mittal Peter Henderson SILM 124 628 0 05 Oct 2023
SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks Alexander Robey Eric Wong Hamed Hassani George J. Pappas AAML 118 255 0 05 Oct 2023
GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts Jiahao Yu Xingwei Lin Zheng Yu Xinyu Xing SILM 185 351 0 19 Sep 2023
Certifying LLM Safety against Adversarial Prompting Aounon Kumar Chirag Agarwal Suraj Srinivas Aaron Jiaxun Li Soheil Feizi Himabindu Lakkaraju AAML 93 193 0 06 Sep 2023
Baseline Defenses for Adversarial Attacks Against Aligned Language Models Neel Jain Avi Schwarzschild Yuxin Wen Gowthami Somepalli John Kirchenbauer Ping Yeh-Chiang Micah Goldblum Aniruddha Saha Jonas Geiping Tom Goldstein AAML 128 400 0 01 Sep 2023
Detecting Language Model Attacks with Perplexity Gabriel Alon Michael Kamfonas AAML 107 227 0 27 Aug 2023
Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilities Maximilian Mozes Xuanli He Bennett Kleinberg Lewis D. Griffin 79 87 0 24 Aug 2023
GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher Youliang Yuan Wenxiang Jiao Wenxuan Wang Jen-tse Huang Pinjia He Shuming Shi Zhaopeng Tu SILM 109 282 0 12 Aug 2023
Universal and Transferable Adversarial Attacks on Aligned Language Models Andy Zou Zifan Wang Nicholas Carlini Milad Nasr J. Zico Kolter Matt Fredrikson 291 1,498 0 27 Jul 2023
Jailbroken: How Does LLM Safety Training Fail? Alexander Wei Nika Haghtalab Jacob Steinhardt 205 995 0 05 Jul 2023
Direct Preference Optimization: Your Language Model is Secretly a Reward Model Rafael Rafailov Archit Sharma E. Mitchell Stefano Ermon Christopher D. Manning Chelsea Finn ALM 387 4,125 0 29 May 2023
Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study Yi Liu Gelei Deng Zhengzi Xu Yuekang Li Yaowen Zheng Ying Zhang Lida Zhao Tianwei Zhang Kailong Wang Yang Liu 79 471 0 23 May 2023
Detecting Hate Speech with GPT-3 Ke-Li Chiu Annie Collins Rohan Alexander AILaw 52 114 0 23 Mar 2021
The Curious Case of Neural Text Degeneration Ari Holtzman Jan Buys Li Du Maxwell Forbes Yejin Choi 193 3,201 0 22 Apr 2019
Hierarchical Neural Story Generation Angela Fan M. Lewis Yann N. Dauphin DiffM 183 1,626 0 13 May 2018