Jailbroken: How Does LLM Safety Training Fail?

5 July 2023

Papers citing "Jailbroken: How Does LLM Safety Training Fail?"

50 / 638 papers shown

Title
The Instinctive Bias: Spurious Images lead to Hallucination in MLLMs Tianyang Han Qing Lian Rui Pan Renjie Pi Jipeng Zhang Shizhe Diao Yong Lin Tong Zhang 75 1 0 06 Feb 2024
Learning to Generate Explainable Stock Predictions using Self-Reflective Large Language Models Kelvin J.L. Koa Yunshan Ma Ritchie Ng Tat-Seng Chua AIFin LLMAG 42 25 0 06 Feb 2024
GUARD: Role-playing to Generate Natural-language Jailbreakings to Test Guideline Adherence of Large Language Models Haibo Jin Ruoxi Chen Andy Zhou Yang Zhang Haohan Wang LLMAG 24 21 0 05 Feb 2024
DeAL: Decoding-time Alignment for Large Language Models James Y. Huang Sailik Sengupta Daniele Bonadiman Yi-An Lai Arshit Gupta Nikolaos Pappas Saab Mansour Katrin Kirchoff Dan Roth 56 29 0 05 Feb 2024
Jailbreaking Attack against Multimodal Large Language Model Zhenxing Niu Haoxuan Ji Xinbo Gao Gang Hua Rong Jin 42 61 0 04 Feb 2024
Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Models Yongshuo Zong Ondrej Bohdal Tingyang Yu Yongxin Yang Timothy M. Hospedales VLM MLLM 57 57 0 03 Feb 2024
On Catastrophic Inheritance of Large Foundation Models Hao Chen Bhiksha Raj Xing Xie Jindong Wang AI4CE 56 12 0 02 Feb 2024
Building Guardrails for Large Language Models Yizhen Dong Ronghui Mu Gao Jin Yi Qi Jinwei Hu Xingyu Zhao Jie Meng Wenjie Ruan Xiaowei Huang OffRL 61 27 0 02 Feb 2024
Vaccine: Perturbation-aware Alignment for Large Language Model Tiansheng Huang Sihao Hu Ling Liu 50 33 0 02 Feb 2024
On Prompt-Driven Safeguarding for Large Language Models Chujie Zheng Fan Yin Hao Zhou Fandong Meng Jie Zhou Kai-Wei Chang Minlie Huang Nanyun Peng AAML 49 47 0 31 Jan 2024
Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks Andy Zhou Bo Li Haohan Wang AAML 49 72 0 30 Jan 2024
Weak-to-Strong Jailbreaking on Large Language Models Xuandong Zhao Xianjun Yang Tianyu Pang Chao Du Lei Li Yu-Xiang Wang William Yang Wang 34 54 0 30 Jan 2024
Security and Privacy Challenges of Large Language Models: A Survey B. Das M. H. Amini Yanzhao Wu PILM ELM 19 103 0 30 Jan 2024
Red-Teaming for Generative AI: Silver Bullet or Security Theater? Michael Feffer Anusha Sinha Wesley Hanwen Deng Zachary Chase Lipton Hoda Heidari AAML 38 67 0 29 Jan 2024
Black-Box Access is Insufficient for Rigorous AI Audits Stephen Casper Carson Ezell Charlotte Siegmann Noam Kolt Taylor Lynn Curtis ... Michael Gerovitch David Bau Max Tegmark David M. Krueger Dylan Hadfield-Menell AAML 34 78 0 25 Jan 2024
MULTIVERSE: Exposing Large Language Model Alignment Problems in Diverse Worlds Xiaolong Jin Zhuo Zhang Xiangyu Zhang 15 3 0 25 Jan 2024
Unmasking and Quantifying Racial Bias of Large Language Models in Medical Report Generation Yifan Yang Xiaoyu Liu Qiao Jin Furong Huang Zhiyong Lu 30 23 0 25 Jan 2024
The Language Barrier: Dissecting Safety Challenges of LLMs in Multilingual Contexts Lingfeng Shen Weiting Tan Sihao Chen Yunmo Chen Jingyu Zhang Haoran Xu Boyuan Zheng Philipp Koehn Daniel Khashabi 34 38 0 23 Jan 2024
Pruning for Protection: Increasing Jailbreak Resistance in Aligned LLMs Without Fine-Tuning Adib Hasan Ileana Rugina Alex Wang AAML 54 22 0 19 Jan 2024
R-Judge: Benchmarking Safety Risk Awareness for LLM Agents Tongxin Yuan Zhiwei He Lingzhong Dong Yiming Wang Ruijie Zhao ... Binglin Zhou Fangqi Li Zhuosheng Zhang Rui Wang Gongshen Liu ELM 34 61 0 18 Jan 2024
Canvil: Designerly Adaptation for LLM-Powered User Experiences K. J. Kevin Feng Q. V. Liao Ziang Xiao Jennifer Wortman Vaughan Amy X. Zhang David W. McDonald 43 16 0 17 Jan 2024
Contrastive Perplexity for Controlled Generation: An Application in Detoxifying Large Language Models T. Klein Moin Nabi 21 1 0 16 Jan 2024
Large Language Models are Null-Shot Learners Pittawat Taveekitworachai Febri Abdullah R. Thawonmas LRM 21 2 0 16 Jan 2024
ML-On-Rails: Safeguarding Machine Learning Models in Software Systems A Case Study Hala Abdelkader Mohamed Abdelrazek Scott Barnett Jean-Guy Schneider Priya Rani Rajesh Vasa 37 3 0 12 Jan 2024
TOFU: A Task of Fictitious Unlearning for LLMs Pratyush Maini Zhili Feng Avi Schwarzschild Zachary Chase Lipton J. Zico Kolter MU CLL 38 142 0 11 Jan 2024
Combating Adversarial Attacks with Multi-Agent Debate Steffi Chern Zhen Fan Andy Liu AAML 37 5 0 11 Jan 2024
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems Tianyu Cui Yanling Wang Chuanpu Fu Yong Xiao Sijia Li ... Junwu Xiong Xinyu Kong Zujie Wen Ke Xu Qi Li 57 56 0 11 Jan 2024
A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicity Andrew Lee Xiaoyan Bai Itamar Pres Martin Wattenberg Jonathan K. Kummerfeld Rada Mihalcea 77 96 0 03 Jan 2024
Detection and Defense Against Prominent Attacks on Preconditioned LLM-Integrated Virtual Assistants C. Chan Daniel Wankit Yip Aysan Esmradi 26 1 0 02 Jan 2024
A Novel Evaluation Framework for Assessing Resilience Against Prompt Injection Attacks in Large Language Models Daniel Wankit Yip Aysan Esmradi C. Chan AAML 28 11 0 02 Jan 2024
Jatmo: Prompt Injection Defense by Task-Specific Finetuning Julien Piet Maha Alrashed Chawin Sitawarin Sizhe Chen Zeming Wei Elizabeth Sun Basel Alomair David A. Wagner AAML SyDa 83 52 0 29 Dec 2023
Large Language Models for Conducting Advanced Text Analytics Information Systems Research Benjamin Ampel Chi-Heng Yang Junjie Hu Hsinchun Chen 33 7 0 27 Dec 2023
A Comprehensive Survey of Attack Techniques, Implementation, and Mitigation Strategies in Large Language Models Aysan Esmradi Daniel Wankit Yip C. Chan AAML 38 11 0 18 Dec 2023
Forbidden Facts: An Investigation of Competing Objectives in Llama-2 Tony T. Wang Miles Wang Kaivu Hariharan Nir Shavit 21 2 0 14 Dec 2023
Distributional Preference Learning: Understanding and Accounting for Hidden Context in RLHF Anand Siththaranjan Cassidy Laidlaw Dylan Hadfield-Menell 31 55 0 13 Dec 2023
Causality Analysis for Evaluating the Security of Large Language Models Wei Zhao Zhe Li Junfeng Sun 26 10 0 13 Dec 2023
AI Control: Improving Safety Despite Intentional Subversion Ryan Greenblatt Buck Shlegeris Kshitij Sachan Fabien Roger 31 40 0 12 Dec 2023
Make Them Spill the Beans! Coercive Knowledge Extraction from (Production) LLMs Zhuo Zhang Guangyu Shen Guanhong Tao Shuyang Cheng Xiangyu Zhang 35 12 0 08 Dec 2023
Analyzing the Inherent Response Tendency of LLMs: Real-World Instructions-Driven Jailbreak Yanrui Du Sendong Zhao Ming Ma Yuhan Chen Bing Qin 26 15 0 07 Dec 2023
Dr. Jekyll and Mr. Hyde: Two Faces of LLMs Matteo Gioele Collu Tom Janssen-Groesbeek Stefanos Koffas Mauro Conti S. Picek 21 1 0 06 Dec 2023
LLM as OS, Agents as Apps: Envisioning AIOS, Agents and the AIOS-Agent Ecosystem Yingqiang Ge Yujie Ren Wenyue Hua Shuyuan Xu Juntao Tan Yongfeng Zhang LLMAG 23 27 0 06 Dec 2023
On the Robustness of Large Multimodal Models Against Image Adversarial Attacks Xuanimng Cui Alejandro Aparcedo Young Kyun Jang Ser-Nam Lim AAML VLM 21 38 0 06 Dec 2023
Tree of Attacks: Jailbreaking Black-Box LLMs Automatically Anay Mehrotra Manolis Zampetakis Paul Kassianik Blaine Nelson Hyrum Anderson Yaron Singer Amin Karbasi 30 204 0 04 Dec 2023
A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly Yifan Yao Jinhao Duan Kaidi Xu Yuanfang Cai Eric Sun Yue Zhang PILM ELM 39 475 0 04 Dec 2023
Universal Jailbreak Backdoors from Poisoned Human Feedback Javier Rando Florian Tramèr 20 60 0 24 Nov 2023
Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agents Zhuosheng Zhang Yao Yao Aston Zhang Xiangru Tang Xinbei Ma ... Yiming Wang Mark B. Gerstein Rui Wang Gongshen Liu Hai Zhao LLMAG LM&Ro LRM 42 53 0 20 Nov 2023
Cognitive Overload: Jailbreaking Large Language Models with Overloaded Logical Thinking Nan Xu Fei Wang Ben Zhou Bangzheng Li Chaowei Xiao Muhao Chen 32 55 0 16 Nov 2023
Bergeron: Combating Adversarial Attacks through a Conscience-Based Alignment Framework Matthew Pisano Peter Ly Abraham Sanders Bingsheng Yao Dakuo Wang T. Strzalkowski Mei Si AAML 24 24 0 16 Nov 2023
Jailbreaking GPT-4V via Self-Adversarial Attacks with System Prompts Yuanwei Wu Xiang Li Yixin Liu Pan Zhou Lichao Sun 13 59 0 15 Nov 2023
Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritization Zhexin Zhang Junxiao Yang Pei Ke Fei Mi Hongning Wang Minlie Huang AAML 28 113 0 15 Nov 2023