Jailbroken: How Does LLM Safety Training Fail?

5 July 2023

Papers citing "Jailbroken: How Does LLM Safety Training Fail?"

50 / 636 papers shown

Title
ReLearn: Unlearning via Learning for Large Language Models Haoming Xu Ningyuan Zhao Liming Yang Sendong Zhao Shumin Deng Mengru Wang Bryan Hooi Nay Oo H. Chen N. Zhang KELM CLL MU 165 0 0 16 Feb 2025
SafeDialBench: A Fine-Grained Safety Benchmark for Large Language Models in Multi-Turn Dialogues with Diverse Jailbreak Attacks Hongye Cao Yanming Wang Sijia Jing Ziyue Peng Zhixin Bai ... Yang Gao Fanyu Meng Xi Yang Chao Deng Junlan Feng AAML 41 0 0 16 Feb 2025
Distraction is All You Need for Multimodal Large Language Model Jailbreaking Zuopeng Yang Jiluan Fan Anli Yan Erdun Gao Xin Lin Tao Li Kanghua mo Changyu Dong AAML 77 0 0 15 Feb 2025
Commercial LLM Agents Are Already Vulnerable to Simple Yet Dangerous Attacks Ang Li Yin Zhou Vethavikashini Chithrra Raghuram Tom Goldstein Micah Goldblum AAML 83 7 0 12 Feb 2025
Leveraging Reasoning with Guidelines to Elicit and Utilize Knowledge for Enhancing Safety Alignment Haoyu Wang Zeyu Qin Li Shen Xueqian Wang Minhao Cheng Dacheng Tao 99 2 0 06 Feb 2025
KDA: A Knowledge-Distilled Attacker for Generating Diverse Prompts to Jailbreak LLMs Buyun Liang Kwan Ho Ryan Chan D. Thaker Jinqi Luo René Vidal AAML 46 0 0 05 Feb 2025
Adversarial ML Problems Are Getting Harder to Solve and to Evaluate Javier Rando Jie Zhang Nicholas Carlini F. Tramèr AAML ELM 61 3 0 04 Feb 2025
Topic-FlipRAG: Topic-Orientated Adversarial Opinion Manipulation Attacks to Retrieval-Augmented Generation Models Jiawei Liu Zhuo Chen Miaokun Chen Fengchang Yu Fan Zhang Xiaofeng Wang Wei Lu Xiaozhong Liu AAML SILM 63 0 0 03 Feb 2025
Breaking Focus: Contextual Distraction Curse in Large Language Models Yue Huang Yanbo Wang Zixiang Xu Chujie Gao Siyuan Wu Jiayi Ye Xiuying Chen Pin-Yu Chen Xuzhi Zhang AAML 48 1 0 03 Feb 2025
"I am bad": Interpreting Stealthy, Universal and Robust Audio Jailbreaks in Audio-Language Models Isha Gupta David Khachaturov Robert D. Mullins AAML AuLLM 65 1 0 02 Feb 2025
Trading Inference-Time Compute for Adversarial Robustness Wojciech Zaremba Evgenia Nitishinskaya Boaz Barak Stephanie Lin Sam Toyer ... Rachel Dias Eric Wallace Kai Y. Xiao Johannes Heidecke Amelia Glaese LRM AAML 96 15 0 31 Jan 2025
The Pitfalls of "Security by Obscurity" And What They Mean for Transparent AI Peter Hall Olivia Mundahl Sunoo Park 76 0 0 30 Jan 2025
When LLM Meets DRL: Advancing Jailbreaking Efficiency via DRL-guided Search Xuan Chen Yuzhou Nie Wenbo Guo Xiangyu Zhang 112 10 0 28 Jan 2025
HumorReject: Decoupling LLM Safety from Refusal Prefix via A Little Humor Zihui Wu Haichang Gao Jiacheng Luo Zhaoxiang Liu 41 0 0 23 Jan 2025
Refining Input Guardrails: Enhancing LLM-as-a-Judge Efficiency Through Chain-of-Thought Fine-Tuning and Alignment Melissa Kazemi Rad Huy Nghiem Andy Luo Sahil Wadhwa Mohammad Sorower Stephen Rawls AAML 93 2 0 22 Jan 2025
Can Safety Fine-Tuning Be More Principled? Lessons Learned from Cybersecurity David Williams-King Linh Le Adam Oberman Yoshua Bengio AAML 56 0 0 19 Jan 2025
Text-Diffusion Red-Teaming of Large Language Models: Unveiling Harmful Behaviors with Proximity Constraints Jonathan Nöther Adish Singla Goran Radanović AAML 57 0 0 14 Jan 2025
Lessons From Red Teaming 100 Generative AI Products Blake Bullwinkel Amanda Minnich Shiven Chawla Gary Lopez Martin Pouliot ... Pete Bryan Ram Shankar Siva Kumar Yonatan Zunger Chang Kawaguchi Mark Russinovich AAML VLM 37 5 0 13 Jan 2025
Safeguarding System Prompts for LLMs Zhifeng Jiang Zhihua Jin Guoliang He AAML SILM 105 1 0 10 Jan 2025
ChatBug: A Common Vulnerability of Aligned LLMs Induced by Chat Templates Fengqing Jiang Zhangchen Xu Luyao Niu Bill Yuchen Lin Radha Poovendran SILM 81 5 0 08 Jan 2025
CALM: Curiosity-Driven Auditing for Large Language Models Xiang Zheng Longxiang Wang Yi Liu Jie Zhang Chao Shen Cong Wang MLAU 55 0 0 06 Jan 2025
Layer-Level Self-Exposure and Patch: Affirmative Token Mitigation for Jailbreak Attack Defense Yang Ouyang Hengrui Gu Shuhang Lin Wenyue Hua Jie Peng B. Kailkhura Tianlong Chen Kaixiong Zhou Kaixiong Zhou AAML 31 1 0 05 Jan 2025
Security Attacks on LLM-based Code Completion Tools Wen Cheng Ke Sun Xinyu Zhang Wei Wang SILM AAML ELM 65 0 0 03 Jan 2025
LLM-Virus: Evolutionary Jailbreak Attack on Large Language Models Miao Yu Fan Zhang Yingjie Zhou Xing Fan Kun Wang Shirui Pan Qingsong Wen AAML 61 0 0 03 Jan 2025
DiffusionAttacker: Diffusion-Driven Prompt Manipulation for LLM Jailbreak Hao Wang Hao Li Junda Zhu Xinyuan Wang C. Pan Minlie Huang Lei Sha 142 0 0 23 Dec 2024
Sim911: Towards Effective and Equitable 9-1-1 Dispatcher Training with an LLM-Enabled Simulation Zirong Chen Elizabeth Chason Noah Mladenovski Erin Wilson Kristin Mullen Stephen Martini Meiyi Ma 85 2 0 22 Dec 2024
Beyond Partisan Leaning: A Comparative Analysis of Political Bias in Large Language Models Kaiqi Yang Hang Li Yucheng Chu Hang Li Tai-Quan Peng Yuping Lin Hui Liu 85 0 0 21 Dec 2024
Jailbreaking? One Step Is Enough! Weixiong Zheng Peijian Zeng Y. Li Hongyan Wu Nankai Lin Jianfei Chen Aimin Yang Yue Zhou AAML 81 0 0 17 Dec 2024
Towards Action Hijacking of Large Language Model-based Agent Yuyang Zhang Kangjie Chen Xudong Jiang Yuxiang Sun Run Wang Lina Wang LLMAG AAML 73 2 0 14 Dec 2024
No Free Lunch for Defending Against Prefilling Attack by In-Context Learning Zhiyu Xue Guangliang Liu Bocheng Chen K. Johnson Ramtin Pedarsani AAML 73 0 0 13 Dec 2024
FlexLLM: Exploring LLM Customization for Moving Target Defense on Black-Box LLMs Against Jailbreak Attacks Bocheng Chen Hanqing Guo Qiben Yan AAML 78 0 0 10 Dec 2024
Improved Large Language Model Jailbreak Detection via Pretrained Embeddings Erick Galinkin Martin Sablotny 76 0 0 02 Dec 2024
Yi-Lightning Technical Report 01. AI : Alan Wake Albert Wang Bei Chen ... Yuxuan Sha Zhaodong Yan Zhiyuan Liu Zirui Zhang Zonghong Dai OSLM 102 3 0 02 Dec 2024
Quantized Delta Weight Is Safety Keeper Yule Liu Zhen Sun Xinlei He Xinyi Huang 93 2 0 29 Nov 2024
Examining Multimodal Gender and Content Bias in ChatGPT-4o Roberto Balestri 84 2 0 28 Nov 2024
Don't Let Your Robot be Harmful: Responsible Robotic Manipulation Minheng Ni Lei Zhang Zhe Chen L. Zhang Wangmeng Zuo 72 1 0 27 Nov 2024
Privacy-Preserving Behaviour of Chatbot Users: Steering Through Trust Dynamics Julia Ive Vishal Yadav Mariia Ignashina Matthew Rand Paulina Bondaronek 76 0 0 26 Nov 2024
Preventing Jailbreak Prompts as Malicious Tools for Cybercriminals: A Cyber Defense Perspective Jean Marie Tshimula Xavier Ndona D'Jeff K. Nkashama Pierre Martin Tardif F. Kabanza Marc Frappier Shengrui Wang SILM 87 0 0 25 Nov 2024
ChemSafetyBench: Benchmarking LLM Safety on Chemistry Domain Haochen Zhao Xiangru Tang Ziran Yang Xiao Han Xuanzhi Feng ... Senhao Cheng Di Jin Yilun Zhao Arman Cohan Mark B. Gerstein ELM 83 1 0 23 Nov 2024
Global Challenge for Safe and Secure LLMs Track 1 Xiaojun Jia Yihao Huang Yang Liu Peng Yan Tan Weng Kuan Yau ... Yan Wang Rick Siow Mong Goh Liangli Zhen Yingjie Zhang Zhe Zhao ELM AILaw 71 0 0 21 Nov 2024
Steering Language Model Refusal with Sparse Autoencoders Kyle O'Brien David Majercak Xavier Fernandes Richard Edgar Jingya Chen Harsha Nori Dean Carignan Eric Horvitz Forough Poursabzi-Sangde LLMSV 67 10 0 18 Nov 2024
JailbreakLens: Interpreting Jailbreak Mechanism in the Lens of Representation and Circuit Zeqing He Zhibo Wang Zhixuan Chu Huiyu Xu Rui Zheng Kui Ren Chun Chen 54 3 0 17 Nov 2024
Developer Perspectives on Licensing and Copyright Issues Arising from Generative AI for Software Development Trevor Stalnaker Nathan Wintersgill Oscar Chaparro Laura A. Heymann M. D. Penta Daniel M German Denys Poshyvanyk 38 0 0 16 Nov 2024
Llama Guard 3 Vision: Safeguarding Human-AI Image Understanding Conversations Jianfeng Chi Ujjwal Karn Hongyuan Zhan Eric Michael Smith Javier Rando Yiming Zhang Kate Plawiak Zacharie Delpierre Coudert Kartikeya Upasani Mahesh Pasupuleti MLLM 3DH 49 20 0 15 Nov 2024
Jailbreak Attacks and Defenses against Multimodal Generative Models: A Survey Xuannan Liu Xing Cui Peipei Li Zekun Li Huaibo Huang Shuhan Xia Miaoxuan Zhang Yueying Zou Ran He AAML 67 8 0 14 Nov 2024
DROJ: A Prompt-Driven Attack against Large Language Models Leyang Hu Boran Wang 34 0 0 14 Nov 2024
New Emerged Security and Privacy of Pre-trained Model: a Survey and Outlook Meng Yang Tianqing Zhu Chi Liu Wanlei Zhou Shui Yu Philip S. Yu AAML ELM PILM 61 1 0 12 Nov 2024
Artificial Intelligence for Biomedical Video Generation Linyuan Li Jianing Qiu Anujit Saha Lin Li Poyuan Li Mengxian He Ziyu Guo Wu Yuan VGen 63 1 0 12 Nov 2024
Beyond the Safety Bundle: Auditing the Helpful and Harmless Dataset Khaoula Chehbouni Jonathan Colaço-Carr Yash More Jackie CK Cheung G. Farnadi 78 0 0 12 Nov 2024
SCAR: Sparse Conditioned Autoencoders for Concept Detection and Steering in LLMs Ruben Härle Felix Friedrich Manuel Brack Bjorn Deiseroth P. Schramowski Kristian Kersting 33 0 0 11 Nov 2024