Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation

10 October 2023

ArXiv (abs)PDF HTML Github (171★)

Papers citing "Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation"

50 / 74 papers shown

Title
Probe before You Talk: Towards Black-box Defense against Backdoor Unalignment for Large Language Models Biao Yi Tiansheng Huang Sishuo Chen Tong Li Zheli Liu Zhixuan Chu Yiming Li AAML 18 9 0 19 Jun 2025
LoX: Low-Rank Extrapolation Robustifies LLM Safety Against Fine-tuning Gabrel J. Perin Runjin Chen Xuxi Chen Nina S. T. Hirata Zhangyang Wang Junyuan Hong AAML 28 0 0 18 Jun 2025
JavelinGuard: Low-Cost Transformer Architectures for LLM Security Yash Datta Sharath Rajasekar 7 0 0 09 Jun 2025
Beyond Jailbreaks: Revealing Stealthier and Broader LLM Security Risks Stemming from Alignment Failures Yukai Zhou Sibei Yang Wenjie Wang AAML 7 0 0 09 Jun 2025
LLMs Caught in the Crossfire: Malware Requests and Jailbreak Challenges Haoyang Li Huan Gao Zhiyuan Zhao Zhiyu Lin Junyu Gao Xuelong Li AAML 16 0 0 09 Jun 2025
AlphaSteer: Learning Refusal Steering with Principled Null-Space Constraint Leheng Sheng Changshuo Shen Weixiang Zhao Junfeng Fang Xiaohao Liu Zhenkai Liang Xiang Wang An Zhang Tat-Seng Chua LLMSV 15 0 0 08 Jun 2025
Adaptive Jailbreaking Strategies Based on the Semantic Understanding Capabilities of Large Language Models Mingyu Yu Wei Wang Y. X. Wei Sujuan Qin AAML 40 0 0 29 May 2025
Seeing the Threat: Vulnerabilities in Vision-Language Models to Adversarial Attack Juan Ren Mark Dras Usman Naseem AAML 71 0 0 28 May 2025
Safety Alignment via Constrained Knowledge Unlearning Zesheng Shi Yucheng Zhou Jing Li MU KELM AAML 68 2 0 24 May 2025
Towards medical AI misalignment: a preliminary study Barbara Puccio Federico Castagna Allan Tucker Pierangelo Veltri 47 0 0 22 May 2025
Exploring Jailbreak Attacks on LLMs through Intent Concealment and Diversion Tiehan Cui Yanxu Mao Peipei Liu Congying Liu Datao You AAML 53 1 0 20 May 2025
Safety Alignment Can Be Not Superficial With Explicit Safety Signals Jianwei Li Jung-Eng Kim AAML 183 1 0 19 May 2025
Safe Delta: Consistently Preserving Safety when Fine-Tuning LLMs on Diverse Datasets Ning Lu Shengcai Liu Jiahao Wu Weiyu Chen Zhirui Zhang Yew-Soon Ong Qi Wang Ke Tang 106 3 0 17 May 2025
JULI: Jailbreak Large Language Models by Self-Introspection Jesson Wang Zhanhao Hu David Wagner 108 0 0 17 May 2025
Multilingual Collaborative Defense for Large Language Models Hongliang Li Jinan Xu Gengping Cui Changhao Guan Fengran Mo Kaiyu Huang AAML 58 0 0 17 May 2025
One Shot Dominance: Knowledge Poisoning Attack on Retrieval-Augmented Generation Systems Zhiyuan Chang Mingyang Li Xiaojun Jia Junjie Wang Yuekai Huang Ziyou Jiang Yebin Liu Qing Wang AAML 83 1 0 15 May 2025
ACE: A Security Architecture for LLM-Integrated App Systems Evan Li Tushin Mallick Evan Rose William K. Robertson Alina Oprea Cristina Nita-Rotaru 109 1 0 29 Apr 2025
Unified Attacks to Large Language Model Watermarks: Spoofing and Scrubbing in Unauthorized Knowledge Distillation Xin Yi Shunfan Zhengc Linlin Wanga Xiaoling Wang Liang He Liang He AAML 482 0 0 24 Apr 2025
Steering the CensorShip: Uncovering Representation Vectors for LLM "Thought" Control Hannah Cyberey David Evans LLMSV 157 3 0 23 Apr 2025
2D-Curri-DPO: Two-Dimensional Curriculum Learning for Direct Preference Optimization Mengyang Li Zhong Zhang 73 1 0 10 Apr 2025
Practical Poisoning Attacks against Retrieval-Augmented Generation Baolei Zhang Yuxiao Chen Minghong Fang Zhuqing Liu Lihai Nie Tong Li Zheli Liu SILM AAML 109 1 0 04 Apr 2025
Adversarial Tokenization Renato Lui Geh Zilei Shao Guy Van den Broeck SILM AAML 152 2 0 04 Mar 2025
Foot-In-The-Door: A Multi-turn Jailbreak for LLMs Zixuan Weng Xiaolong Jin Jinyuan Jia Xinsong Zhang AAML 381 1 0 27 Feb 2025
GuidedBench: Equipping Jailbreak Evaluation with Guidelines Ruixuan Huang Xunguang Wang Zongjie Li Daoyuan Wu Shuai Wang ALM ELM 129 0 0 24 Feb 2025
Adversarial Prompt Evaluation: Systematic Benchmarking of Guardrails Against Prompt Input Attacks on LLMs Giulio Zizzo Giandomenico Cornacchia Kieran Fraser Muhammad Zaid Hameed Ambrish Rawat Beat Buesser Mark Purcell Pin-Yu Chen P. Sattigeri Kush R. Varshney AAML 104 5 0 24 Feb 2025
Intrinsic Model Weaknesses: How Priming Attacks Unveil Vulnerabilities in Large Language Models Yuyi Huang Runzhe Zhan Derek F. Wong Lidia S. Chao Ailin Tao AAML SyDa ELM 69 0 0 23 Feb 2025
A generative approach to LLM harmfulness detection with special red flag tokens Sophie Xhonneux David Dobre Mehrnaz Mohfakhami Leo Schwinn Gauthier Gidel 178 2 0 22 Feb 2025
SafeInt: Shielding Large Language Models from Jailbreak Attacks via Safety-Aware Representation Intervention Jiaqi Wu Chen Chen Chunyan Hou Xiaojie Yuan AAML 130 0 0 21 Feb 2025
A Mousetrap: Fooling Large Reasoning Models for Jailbreak with Chain of Iterative Chaos Yang Yao Xuan Tong Ruofan Wang Yixu Wang Lujundong Li Liang Liu Yan Teng Yun Wang LRM 84 7 0 19 Feb 2025
Equilibrate RLHF: Towards Balancing Helpfulness-Safety Trade-off in Large Language Models Yingshui Tan Yilei Jiang Yongbin Li Qingbin Liu Xingyuan Bu Wenbo Su Xiangyu Yue Xiaoyong Zhu Bo Zheng ALM 150 6 0 17 Feb 2025
DELMAN: Dynamic Defense Against Large Language Model Jailbreaking with Model Editing Yi Wang Fenghua Weng Shangshang Yang Zhan Qin Minlie Huang Wenjie Wang KELM AAML 117 1 0 17 Feb 2025
FLAME: Flexible LLM-Assisted Moderation Engine Ivan Bakulin Ilia Kopanichuk Iaroslav Bespalov Nikita Radchenko V. Shaposhnikov Dmitry V. Dylov Ivan Oseledets 170 0 0 13 Feb 2025
Trustworthy AI: Safety, Bias, and Privacy -- A Survey Xingli Fang Jianwei Li Varun Mulchandani Jung-Eun Kim 82 0 0 11 Feb 2025
Confidence Elicitation: A New Attack Vector for Large Language Models Brian Formento Chuan-Sheng Foo See-Kiong Ng AAML 262 0 0 07 Feb 2025
When LLM Meets DRL: Advancing Jailbreaking Efficiency via DRL-guided Search Xuan Chen Yuzhou Nie Wenbo Guo Xiangyu Zhang 202 18 0 28 Jan 2025
Refining Input Guardrails: Enhancing LLM-as-a-Judge Efficiency Through Chain-of-Thought Fine-Tuning and Alignment Melissa Kazemi Rad Huy Nghiem Andy Luo Sahil Wadhwa Mohammad Sorower Stephen Rawls AAML 140 5 0 22 Jan 2025
Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templates Kaifeng Lyu Haoyu Zhao Xinran Gu Dingli Yu Anirudh Goyal Sanjeev Arora ALM 131 59 0 20 Jan 2025
ChatBug: A Common Vulnerability of Aligned LLMs Induced by Chat Templates Fengqing Jiang Zhangchen Xu Luyao Niu Bill Yuchen Lin Radha Poovendran SILM 118 11 0 08 Jan 2025
Layer-Level Self-Exposure and Patch: Affirmative Token Mitigation for Jailbreak Attack Defense Yang Ouyang Hengrui Gu Shuhang Lin Wenyue Hua Jie Peng B. Kailkhura Tianlong Chen Kaixiong Zhou Kaixiong Zhou AAML 107 3 0 05 Jan 2025
Beyond the Safety Bundle: Auditing the Helpful and Harmless Dataset Khaoula Chehbouni Jonathan Colaço-Carr Yash More Jackie CK Cheung G. Farnadi 173 1 0 12 Nov 2024
Diversity Helps Jailbreak Large Language Models Weiliang Zhao Daniel Ben-Levi Wei Hao Junfeng Yang Chengzhi Mao AAML 481 1 0 06 Nov 2024
AmpleGCG-Plus: A Strong Generative Model of Adversarial Suffixes to Jailbreak LLMs with Higher Success Rates in Fewer Attempts Vishal Kumar Zeyi Liao Jaylen Jones Huan Sun AAML 116 3 0 29 Oct 2024
ADVLLM: Iterative Self-Tuning LLMs for Enhanced Jailbreaking Capabilities Chung-En Sun Xiaodong Liu Weiwei Yang Tsui-Wei Weng Hao Cheng Aidan San Michel Galley J. Gao 132 2 0 24 Oct 2024
On the Role of Attention Heads in Large Language Model Safety Zhenhong Zhou Haiyang Yu Xinghua Zhang Rongwu Xu Fei Huang Kun Wang Yang Liu Sihang Li Yongbin Li 147 10 0 17 Oct 2024
MACPO: Weak-to-Strong Alignment via Multi-Agent Contrastive Preference Optimization Yougang Lyu Lingyong Yan Zihan Wang Dawei Yin Pengjie Ren Maarten de Rijke Zhaochun Ren 148 10 0 10 Oct 2024
Root Defence Strategies: Ensuring Safety of LLM at the Decoding Level Xinyi Zeng Yuying Shang Yutao Zhu Jingyuan Zhang Yu Tian AAML 491 4 0 09 Oct 2024
Cheating Automatic LLM Benchmarks: Null Models Achieve High Win Rates Xiaosen Zheng Tianyu Pang Chao Du Qian Liu Jing Jiang Min Lin 89 13 0 09 Oct 2024
Surgical, Cheap, and Flexible: Mitigating False Refusal in Language Models via Single Vector Ablation Xinpeng Wang Chengzhi Hu Paul Röttger Barbara Plank 147 11 0 04 Oct 2024
Endless Jailbreaks with Bijection Learning Brian R. Y. Huang Maximilian Li Leonard Tang AAML 150 8 0 02 Oct 2024
An Adversarial Perspective on Machine Unlearning for AI Safety Jakub Łucki Boyi Wei Yangsibo Huang Peter Henderson F. Tramèr Javier Rando MU AAML 200 53 0 26 Sep 2024