Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations

10 October 2023

Zeming Wei

Papers citing "Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations"

41 / 191 papers shown

Title
Leveraging the Context through Multi-Round Interactions for Jailbreaking Attacks Yixin Cheng Markos Georgopoulos V. Cevher Grigorios G. Chrysos AAML 27 15 0 14 Feb 2024
SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decoding Zhangchen Xu Fengqing Jiang Luyao Niu Jinyuan Jia Bill Yuchen Lin Radha Poovendran AAML 131 86 0 14 Feb 2024
COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllability Xing-ming Guo Fangxu Yu Huan Zhang Lianhui Qin Bin Hu AAML 117 70 0 13 Feb 2024
Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast Xiangming Gu Xiaosen Zheng Tianyu Pang Chao Du Qian Liu Ye Wang Jing Jiang Min-Bin Lin LLMAG LM&Ro 37 49 0 13 Feb 2024
StruQ: Defending Against Prompt Injection with Structured Queries Sizhe Chen Julien Piet Chawin Sitawarin David Wagner SILM AAML 30 67 0 09 Feb 2024
Fight Back Against Jailbreaking via Prompt Adversarial Tuning Yichuan Mo Yuji Wang Zeming Wei Yisen Wang AAML SILM 49 25 0 09 Feb 2024
In-Context Learning Can Re-learn Forbidden Tasks Sophie Xhonneux David Dobre Jian Tang Gauthier Gidel Dhanya Sridhar 24 0 0 08 Feb 2024
Prioritizing Safeguarding Over Autonomy: Risks of LLM Agents for Science Xiangru Tang Qiao Jin Kunlun Zhu Tongxin Yuan Yichi Zhang ... Jian Tang Zhuosheng Zhang Arman Cohan Zhiyong Lu Mark B. Gerstein LLMAG ELM 25 41 0 06 Feb 2024
GUARD: Role-playing to Generate Natural-language Jailbreakings to Test Guideline Adherence of Large Language Models Haibo Jin Ruoxi Chen Andy Zhou Yang Zhang Haohan Wang LLMAG 24 21 0 05 Feb 2024
Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks Andy Zhou Bo Li Haohan Wang AAML 49 74 0 30 Jan 2024
Weak-to-Strong Jailbreaking on Large Language Models Xuandong Zhao Xianjun Yang Tianyu Pang Chao Du Lei Li Yu-Xiang Wang William Y. Wang 34 54 0 30 Jan 2024
Red-Teaming for Generative AI: Silver Bullet or Security Theater? Michael Feffer Anusha Sinha Wesley Hanwen Deng Zachary Chase Lipton Hoda Heidari AAML 38 67 0 29 Jan 2024
How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs Yi Zeng Hongpeng Lin Jingwen Zhang Diyi Yang Ruoxi Jia Weiyan Shi 18 256 0 12 Jan 2024
Intention Analysis Makes LLMs A Good Jailbreak Defender Yuqi Zhang Liang Ding Lefei Zhang Dacheng Tao LLMSV 30 16 0 12 Jan 2024
Universal Vulnerabilities in Large Language Models: Backdoor Attacks for In-context Learning Shuai Zhao Meihuizi Jia Anh Tuan Luu Fengjun Pan Jinming Wen AAML 31 36 0 11 Jan 2024
MLLM-Protector: Ensuring MLLM's Safety without Hurting Performance Renjie Pi Tianyang Han Jianshu Zhang Yueqi Xie Rui Pan Qing Lian Hanze Dong Jipeng Zhang Tong Zhang AAML 23 59 0 05 Jan 2024
The Art of Defending: A Systematic Evaluation and Analysis of LLM Defense Strategies on Safety and Over-Defensiveness Neeraj Varshney Pavel Dolin Agastya Seth Chitta Baral AAML ELM 25 47 0 30 Dec 2023
Jatmo: Prompt Injection Defense by Task-Specific Finetuning Julien Piet Maha Alrashed Chawin Sitawarin Sizhe Chen Zeming Wei Elizabeth Sun Basel Alomair David Wagner AAML SyDa 83 53 0 29 Dec 2023
Safety Alignment in NLP Tasks: Weakly Aligned Summarization as an In-Context Attack Yu Fu Yufei Li Wen Xiao Cong Liu Yue Dong AAML 42 5 0 12 Dec 2023
A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly Yifan Yao Jinhao Duan Kaidi Xu Yuanfang Cai Eric Sun Yue Zhang PILM ELM 44 475 0 04 Dec 2023
MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models Xin Liu Yichen Zhu Jindong Gu Yunshi Lan Chao Yang Yu Qiao 30 84 0 29 Nov 2023
How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMs Haoqin Tu Chenhang Cui Zijun Wang Yiyang Zhou Bingchen Zhao Junlin Han Wangchunshu Zhou Huaxiu Yao Cihang Xie MLLM 60 71 0 27 Nov 2023
Hijacking Large Language Models via Adversarial In-Context Learning Yao Qiang Xiangyu Zhou Dongxiao Zhu 32 32 0 16 Nov 2023
Cognitive Overload: Jailbreaking Large Language Models with Overloaded Logical Thinking Nan Xu Fei Wang Ben Zhou Bangzheng Li Chaowei Xiao Muhao Chen 32 55 0 16 Nov 2023
Stealthy and Persistent Unalignment on Large Language Models via Backdoor Injections Yuanpu Cao Bochuan Cao Jinghui Chen 34 24 0 15 Nov 2023
Fake Alignment: Are LLMs Really Aligned Well? Yixu Wang Yan Teng Kexin Huang Chengqi Lyu Songyang Zhang Wenwei Zhang Xingjun Ma Yu-Gang Jiang Yu Qiao Yingchun Wang 35 16 0 10 Nov 2023
DeepInception: Hypnotize Large Language Model to Be Jailbreaker Xuan Li Zhanke Zhou Jianing Zhu Jiangchao Yao Tongliang Liu Bo Han 50 152 0 06 Nov 2023
AutoDAN: Interpretable Gradient-Based Adversarial Attacks on Large Language Models Sicheng Zhu Ruiyi Zhang Bang An Gang Wu Joe Barrow Zichao Wang Furong Huang A. Nenkova Tong Sun SILM AAML 30 41 0 23 Oct 2023
Privacy in Large Language Models: Attacks, Defenses and Future Directions Haoran Li Yulin Chen Jinglong Luo Yan Kang Xiaojin Zhang Qi Hu Chunkit Chan Yangqiu Song PILM 48 42 0 16 Oct 2023
Defending Against Alignment-Breaking Attacks via Robustly Aligned LLM Bochuan Cao Yu Cao Lu Lin Jinghui Chen AAML 36 135 0 18 Sep 2023
Enhancing Adversarial Attacks: The Similar Target Method Shuo Zhang Ziruo Wang Zikai Zhou Huanran Chen AAML 54 1 0 21 Aug 2023
Robust Classification via a Single Diffusion Model Huanran Chen Yinpeng Dong Zhengyi Wang X. Yang Chen-Dong Duan Hang Su Jun Zhu 82 56 0 24 May 2023
$k$ NN Prompting: Beyond-Context Learning with Calibration-Free Nearest Neighbor Inference Benfeng Xu Quan Wang Zhendong Mao Yajuan Lyu Qiaoqiao She Yongdong Zhang 104 52 0 24 Mar 2023
Rethinking Model Ensemble in Transfer-based Adversarial Attacks Huanran Chen Yichi Zhang Yinpeng Dong Xiao Yang Hang Su Junyi Zhu AAML 28 56 0 16 Mar 2023
The Learnability of In-Context Learning Noam Wies Yoav Levine Amnon Shashua 122 92 0 14 Mar 2023
Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned Deep Ganguli Liane Lovitt John Kernion Amanda Askell Yuntao Bai ... Nicholas Joseph Sam McCandlish C. Olah Jared Kaplan Jack Clark 231 446 0 23 Aug 2022
Instruction Induction: From Few Examples to Natural Language Task Descriptions Or Honovich Uri Shaham Samuel R. Bowman Omer Levy ELM LRM 120 137 0 22 May 2022
Diffusion Models for Adversarial Purification Weili Nie Brandon Guo Yujia Huang Chaowei Xiao Arash Vahdat Anima Anandkumar WIGM 218 419 0 16 May 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 339 12,003 0 04 Mar 2022
Gradient-based Adversarial Attacks against Text Transformers Chuan Guo Alexandre Sablayrolles Hervé Jégou Douwe Kiela SILM 106 227 0 15 Apr 2021
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 299 6,984 0 20 Apr 2018