Jailbroken: How Does LLM Safety Training Fail?

5 July 2023

Papers citing "Jailbroken: How Does LLM Safety Training Fail?"

40 / 640 papers shown

Title
FuzzLLM: A Novel and Universal Fuzzing Framework for Proactively Discovering Jailbreak Vulnerabilities in Large Language Models Dongyu Yao Jianshu Zhang Ian G. Harris Marcel Carlsson 29 30 0 11 Sep 2023
Down the Toxicity Rabbit Hole: A Novel Framework to Bias Audit Large Language Models Arka Dutta Adel Khorramrouz Sujan Dutta Ashiqur R. KhudaBukhsh 22 0 0 08 Sep 2023
Certifying LLM Safety against Adversarial Prompting Aounon Kumar Chirag Agarwal Suraj Srinivas Aaron Jiaxun Li S. Feizi Himabindu Lakkaraju AAML 27 165 0 06 Sep 2023
Demystifying RCE Vulnerabilities in LLM-Integrated Apps Tong Liu Zizhuang Deng Guozhu Meng Yuekang Li Kai Chen SILM 44 19 0 06 Sep 2023
Open Sesame! Universal Black Box Jailbreaking of Large Language Models Raz Lapid Ron Langberg Moshe Sipper AAML 27 105 0 04 Sep 2023
Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models Yue Zhang Yafu Li Leyang Cui Deng Cai Lemao Liu ... Longyue Wang A. Luu Wei Bi Freda Shi Shuming Shi RALM LRM HILM 46 522 0 03 Sep 2023
Let the Models Respond: Interpreting Language Model Detoxification Through the Lens of Prompt Dependence Daniel Scalena Gabriele Sarti Malvina Nissim Elisabetta Fersini 24 0 0 01 Sep 2023
Image Hijacks: Adversarial Images can Control Generative Models at Runtime Luke Bailey Euan Ong Stuart J. Russell Scott Emmons VLM MLLM 30 79 0 01 Sep 2023
Cultural Alignment in Large Language Models: An Explanatory Analysis Based on Hofstede's Cultural Dimensions Reem I. Masoud Ziquan Liu Martin Ferianc Philip C. Treleaven Miguel R. D. Rodrigues 27 50 0 25 Aug 2023
Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMs Yuxia Wang Haonan Li Xudong Han Preslav Nakov Timothy Baldwin 52 103 0 25 Aug 2023
Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilities Maximilian Mozes Xuanli He Bennett Kleinberg Lewis D. Griffin 39 78 0 24 Aug 2023
Sparks of Large Audio Models: A Survey and Outlook S. Latif Moazzam Shoukat Fahad Shamshad Muhammad Usama Yi Ren ... Wenwu Wang Xulong Zhang Roberto Togneri Min Zhang Björn W. Schuller LM&MA AuLLM 33 38 0 24 Aug 2023
Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questions Pouya Pezeshkpour Estevam R. Hruschka LRM 20 131 0 22 Aug 2023
Using Large Language Models for Cybersecurity Capture-The-Flag Challenges and Certification Questions W. Tann Yuancheng Liu Jun Heng Sim C. Seah E. Chang ELM 25 31 0 21 Aug 2023
GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher Youliang Yuan Wenxiang Jiao Wenxuan Wang Jen-tse Huang Pinjia He Shuming Shi Zhaopeng Tu SILM 76 234 0 12 Aug 2023
"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models Xinyue Shen Zhenpeng Chen Michael Backes Yun Shen Yang Zhang SILM 40 249 0 07 Aug 2023
AgentBench: Evaluating LLMs as Agents Xiao Liu Hao Yu Hanchen Zhang Yifan Xu Xuanyu Lei ... Yu-Chuan Su Huan Sun Minlie Huang Yuxiao Dong Jie Tang ELM LLMAG 37 261 0 07 Aug 2023
XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models Paul Röttger Hannah Rose Kirk Bertie Vidgen Giuseppe Attanasio Federico Bianchi Dirk Hovy ALM ELM AILaw 27 127 0 02 Aug 2023
Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injection Jun Yan Vikas Yadav Shiyang Li Lichang Chen Zheng Tang Hai Wang Vijay Srinivasan Xiang Ren Hongxia Jin SILM 28 82 0 31 Jul 2023
Deception Abilities Emerged in Large Language Models Thilo Hagendorff LLMAG 35 76 0 31 Jul 2023
Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback Stephen Casper Xander Davies Claudia Shi T. Gilbert Jérémy Scheurer ... Erdem Biyik Anca Dragan David M. Krueger Dorsa Sadigh Dylan Hadfield-Menell ALM OffRL 52 473 0 27 Jul 2023
Universal and Transferable Adversarial Attacks on Aligned Language Models Andy Zou Zifan Wang Nicholas Carlini Milad Nasr J. Zico Kolter Matt Fredrikson 92 1,278 0 27 Jul 2023
Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models Erfan Shayegani Yue Dong Nael B. Abu-Ghazaleh 47 128 0 26 Jul 2023
LLM Censorship: A Machine Learning Challenge or a Computer Security Problem? David Glukhov Ilia Shumailov Y. Gal Nicolas Papernot Vardan Papyan AAML ELM 30 57 0 20 Jul 2023
Latent Jailbreak: A Benchmark for Evaluating Text Safety and Output Robustness of Large Language Models Huachuan Qiu Shuai Zhang Anqi Li Hongliang He Zhenzhong Lan ALM 44 48 0 17 Jul 2023
Effective Prompt Extraction from Language Models Yiming Zhang Nicholas Carlini Daphne Ippolito MIACV SILM 38 36 0 13 Jul 2023
A Comprehensive Overview of Large Language Models Humza Naveed Asad Ullah Khan Shi Qiu Muhammad Saqib Saeed Anwar Muhammad Usman Naveed Akhtar Nick Barnes Ajmal Mian OffRL 70 529 0 12 Jul 2023
Visual Adversarial Examples Jailbreak Aligned Large Language Models Xiangyu Qi Kaixuan Huang Ashwinee Panda Peter Henderson Mengdi Wang Prateek Mittal AAML 25 138 0 22 Jun 2023
Explore, Establish, Exploit: Red Teaming Language Models from Scratch Stephen Casper Jason Lin Joe Kwon Gatlen Culp Dylan Hadfield-Menell AAML 8 83 0 15 Jun 2023
Science in the Era of ChatGPT, Large Language Models and Generative AI: Challenges for Research Ethics and How to Respond Evangelos Pournaras 33 4 0 24 May 2023
Tricking LLMs into Disobedience: Formalizing, Analyzing, and Detecting Jailbreaks Abhinav Rao S. Vashistha Atharva Naik Somak Aditya Monojit Choudhury 38 17 0 24 May 2023
The Machine Psychology of Cooperation: Can GPT models operationalise prompts for altruism, cooperation, competitiveness and selfishness in economic games? S. Phelps Y. Russell 35 15 0 13 May 2023
Analyzing And Editing Inner Mechanisms Of Backdoored Language Models Max Lamparth Anka Reuel KELM 36 10 0 24 Feb 2023
Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection Kai Greshake Sahar Abdelnabi Shailesh Mishra C. Endres Thorsten Holz Mario Fritz SILM 49 439 0 23 Feb 2023
On the Impossible Safety of Large AI Models El-Mahdi El-Mhamdi Sadegh Farhadkhani R. Guerraoui Nirupam Gupta L. Hoang Rafael Pinot Sébastien Rouault John Stephan 30 31 0 30 Sep 2022
Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned Deep Ganguli Liane Lovitt John Kernion Amanda Askell Yuntao Bai ... Nicholas Joseph Sam McCandlish C. Olah Jared Kaplan Jack Clark 231 446 0 23 Aug 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 339 12,003 0 04 Mar 2022
Challenges in Detoxifying Language Models Johannes Welbl Amelia Glaese J. Uesato Sumanth Dathathri John F. J. Mellor Lisa Anne Hendricks Kirsty Anderson Pushmeet Kohli Ben Coppin Po-Sen Huang LM&MA 250 193 0 15 Sep 2021
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown D. Song Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 290 1,824 0 14 Dec 2020
Fine-Tuning Language Models from Human Preferences Daniel M. Ziegler Nisan Stiennon Jeff Wu Tom B. Brown Alec Radford Dario Amodei Paul Christiano G. Irving ALM 301 1,610 0 18 Sep 2019