Jailbroken: How Does LLM Safety Training Fail?

5 July 2023

Papers citing "Jailbroken: How Does LLM Safety Training Fail?"

50 / 638 papers shown

Title
Alignment is not sufficient to prevent large language models from generating harmful information: A psychoanalytic perspective Zi Yin Wei Ding Jia Liu 27 1 0 14 Nov 2023
SimpleSafetyTests: a Test Suite for Identifying Critical Safety Risks in Large Language Models Bertie Vidgen Nino Scherrer Hannah Rose Kirk Rebecca Qian Anand Kannappan Scott A. Hale Paul Röttger ALM ELM 32 27 0 14 Nov 2023
A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily Peng Ding Jun Kuang Dan Ma Xuezhi Cao Yunsen Xian Jiajun Chen Shujian Huang AAML 30 96 0 14 Nov 2023
Generalization Analogies: A Testbed for Generalizing AI Oversight to Hard-To-Measure Domains Joshua Clymer Garrett Baker Rohan Subramani Sam Wang 22 6 0 13 Nov 2023
Flames: Benchmarking Value Alignment of LLMs in Chinese Kexin Huang Xiangyang Liu Qianyu Guo Tianxiang Sun Jiawei Sun ... Yixu Wang Yan Teng Xipeng Qiu Yingchun Wang Dahua Lin ALM 35 9 0 12 Nov 2023
Fake Alignment: Are LLMs Really Aligned Well? Yixu Wang Yan Teng Kexin Huang Chengqi Lyu Songyang Zhang Wenwei Zhang Xingjun Ma Yu-Gang Jiang Yu Qiao Yingchun Wang 35 15 0 10 Nov 2023
FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts Yichen Gong Delong Ran Jinyuan Liu Conglei Wang Tianshuo Cong Anyu Wang Sisi Duan Xiaoyun Wang MLLM 129 118 0 09 Nov 2023
A Survey of Large Language Models in Medicine: Progress, Application, and Challenge Hongjian Zhou Fenglin Liu Boyang Gu Xinyu Zou Jinfa Huang ... Yefeng Zheng Lei A. Clifton Zheng Li Fenglin Liu David A. Clifton LM&MA 33 107 0 09 Nov 2023
Frontier Language Models are not Robust to Adversarial Arithmetic, or "What do I need to say so you agree 2+2=5? C. D. Freeman Laura J. Culp Aaron T Parisi Maxwell Bileschi Gamaleldin F. Elsayed ... Peter J. Liu Roman Novak Yundi Qian Noah Fiedel Jascha Narain Sohl-Dickstein AAML 33 2 0 08 Nov 2023
Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulation Rusheb Shah Quentin Feuillade--Montixi Soroush Pour Arush Tagade Stephen Casper Javier Rando 26 123 0 06 Nov 2023
DeepInception: Hypnotize Large Language Model to Be Jailbreaker Xuan Li Zhanke Zhou Jianing Zhu Jiangchao Yao Tongliang Liu Bo Han 47 151 0 06 Nov 2023
Can LLMs Follow Simple Rules? Norman Mu Sarah Chen Zifan Wang Sizhe Chen David Karamardian Lulwa Aljeraisy Basel Alomair Dan Hendrycks David A. Wagner ALM 25 27 0 06 Nov 2023
Tensor Trust: Interpretable Prompt Injection Attacks from an Online Game Sam Toyer Olivia Watkins Ethan Mendes Justin Svegliato Luke Bailey ... Karim Elmaaroufi Pieter Abbeel Trevor Darrell Alan Ritter Stuart J. Russell 21 71 0 02 Nov 2023
The Alignment Ceiling: Objective Mismatch in Reinforcement Learning from Human Feedback Nathan Lambert Roberto Calandra ALM 29 31 0 31 Oct 2023
RAIFLE: Reconstruction Attacks on Interaction-based Federated Learning with Adversarial Data Manipulation Dzung Pham Shreyas Kulkarni Amir Houmansadr 33 0 0 29 Oct 2023
Ignore This Title and HackAPrompt: Exposing Systemic Vulnerabilities of LLMs through a Global Scale Prompt Hacking Competition Sander Schulhoff Jeremy Pinto Anaum Khan Louis-Franccois Bouchard Chenglei Si Svetlina Anati Valen Tagliabue Anson Liu Kost Christopher Carnahan Jordan L. Boyd-Graber SILM 37 41 0 24 Oct 2023
Self-Guard: Empower the LLM to Safeguard Itself Zezhong Wang Fangkai Yang Lu Wang Pu Zhao Hongru Wang Liang Chen Qingwei Lin Kam-Fai Wong 80 29 0 24 Oct 2023
The Janus Interface: How Fine-Tuning in Large Language Models Amplifies the Privacy Risks Xiaoyi Chen Siyuan Tang Rui Zhu Shijun Yan Lei Jin Zihao Wang Liya Su Zhikun Zhang Xiaofeng Wang Haixu Tang AAML PILM 21 17 0 24 Oct 2023
AutoDAN: Interpretable Gradient-Based Adversarial Attacks on Large Language Models Sicheng Zhu Ruiyi Zhang Bang An Gang Wu Joe Barrow Zichao Wang Furong Huang A. Nenkova Tong Sun SILM AAML 30 40 0 23 Oct 2023
Formalizing and Benchmarking Prompt Injection Attacks and Defenses Yupei Liu Yuqi Jia Runpeng Geng Jinyuan Jia Neil Zhenqiang Gong SILM LLMAG 27 62 0 19 Oct 2023
Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting Melanie Sclar Yejin Choi Yulia Tsvetkov Alane Suhr 38 304 0 17 Oct 2023
NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails Traian Rebedea R. Dinu Makesh Narsimhan Sreedhar Christopher Parisien Jonathan Cohen KELM 19 132 0 16 Oct 2023
Privacy in Large Language Models: Attacks, Defenses and Future Directions Haoran Li Yulin Chen Jinglong Luo Yan Kang Xiaojin Zhang Qi Hu Chunkit Chan Yangqiu Song PILM 48 42 0 16 Oct 2023
Prompt Packer: Deceiving LLMs through Compositional Instruction with Hidden Attacks Shuyu Jiang Xingshu Chen Rui Tang 24 22 0 16 Oct 2023
$Is Certifying $\ell_p$ Robustness Still Worthwhile?$ Is Certifying $\ell_p$ Robustness Still Worthwhile? Ravi Mangal Klas Leino Zifan Wang Kai Hu Weicheng Yu Corina S. Pasareanu Anupam Datta Matt Fredrikson AAML OOD 33 1 0 13 Oct 2023
Jailbreaking Black Box Large Language Models in Twenty Queries Patrick Chao Alexander Robey Yan Sun Hamed Hassani George J. Pappas Eric Wong AAML 59 572 0 12 Oct 2023
Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation Yangsibo Huang Samyak Gupta Mengzhou Xia Kai Li Danqi Chen AAML 32 268 0 10 Oct 2023
Multilingual Jailbreak Challenges in Large Language Models Yue Deng Wenxuan Zhang Sinno Jialin Pan Lidong Bing AAML 36 113 0 10 Oct 2023
Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations Zeming Wei Yifei Wang Ang Li Yichuan Mo Yisen Wang 48 236 0 10 Oct 2023
Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! Xiangyu Qi Yi Zeng Tinghao Xie Pin-Yu Chen Ruoxi Jia Prateek Mittal Peter Henderson SILM 70 525 0 05 Oct 2023
SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks Alexander Robey Eric Wong Hamed Hassani George J. Pappas AAML 43 215 0 05 Oct 2023
Misusing Tools in Large Language Models With Visual Adversarial Examples Xiaohan Fu Zihan Wang Shuheng Li Rajesh K. Gupta Niloofar Mireshghallah Taylor Berg-Kirkpatrick Earlence Fernandes AAML 29 24 0 04 Oct 2023
Low-Resource Languages Jailbreak GPT-4 Zheng-Xin Yong Cristina Menghini Stephen H. Bach SILM 31 170 0 03 Oct 2023
Jailbreaker in Jail: Moving Target Defense for Large Language Models Bocheng Chen Advait Paliwal Qiben Yan AAML 37 14 0 03 Oct 2023
AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models Xiaogeng Liu Nan Xu Muhao Chen Chaowei Xiao SILM 32 261 0 03 Oct 2023
Can Language Models be Instructed to Protect Personal Information? Yang Chen Ethan Mendes Sauvik Das Wei-ping Xu Alan Ritter PILM 21 34 0 03 Oct 2023
Large Language Models Cannot Self-Correct Reasoning Yet Jie Huang Xinyun Chen Swaroop Mishra Huaixiu Steven Zheng Adams Wei Yu Xinying Song Denny Zhou ReLM LRM 35 421 0 03 Oct 2023
On the Safety of Open-Sourced Large Language Models: Does Alignment Really Prevent Them From Being Misused? Hangfan Zhang Zhimeng Guo Huaisheng Zhu Bochuan Cao Lu Lin Jinyuan Jia Jinghui Chen Di Wu 78 23 0 02 Oct 2023
LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examples Jia-Yu Yao Kun-Peng Ning Zhen-Hui Liu Munan Ning Li Yuan HILM LRM AAML 26 169 0 02 Oct 2023
LoRA ensembles for large language model fine-tuning Xi Wang Laurence Aitchison Maja Rudolph UQCV 34 34 0 29 Sep 2023
Warfare:Breaking the Watermark Protection of AI-Generated Content Guanlin Li Yifei Chen Jie Zhang Shangwei Guo Shangwei Guo Tianwei Zhang Jiwei Li Tianwei Zhang WIGM 58 3 0 27 Sep 2023
Can LLM-Generated Misinformation Be Detected? Canyu Chen Kai Shu DeLMO 39 158 0 25 Sep 2023
LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset Lianmin Zheng Wei-Lin Chiang Ying Sheng Tianle Li Siyuan Zhuang ... Zi Lin Eric P. Xing Joseph E. Gonzalez Ion Stoica Haotong Zhang 27 178 0 21 Sep 2023
How Robust is Google's Bard to Adversarial Image Attacks? Yinpeng Dong Huanran Chen Jiawei Chen Zhengwei Fang X. Yang Yichi Zhang Yu Tian Hang Su Jun Zhu AAML 36 102 0 21 Sep 2023
GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts Jiahao Yu Xingwei Lin Zheng Yu Xinyu Xing SILM 117 301 0 19 Sep 2023
Understanding Catastrophic Forgetting in Language Models via Implicit Inference Suhas Kotha Jacob Mitchell Springer Aditi Raghunathan CLL 42 57 0 18 Sep 2023
Defending Against Alignment-Breaking Attacks via Robustly Aligned LLM Bochuan Cao Yu Cao Lu Lin Jinghui Chen AAML 36 133 0 18 Sep 2023
Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructions Federico Bianchi Mirac Suzgun Giuseppe Attanasio Paul Röttger Dan Jurafsky Tatsunori Hashimoto James Zou ALM LM&MA LRM 34 178 0 14 Sep 2023
FuzzLLM: A Novel and Universal Fuzzing Framework for Proactively Discovering Jailbreak Vulnerabilities in Large Language Models Dongyu Yao Jianshu Zhang Ian G. Harris Marcel Carlsson 24 30 0 11 Sep 2023
Down the Toxicity Rabbit Hole: A Novel Framework to Bias Audit Large Language Models Arka Dutta Adel Khorramrouz Sujan Dutta Ashiqur R. KhudaBukhsh 22 0 0 08 Sep 2023