Talking Nonsense: Probing Large Language Models' Understanding of Adversarial Gibberish Inputs

26 April 2024

Papers citing "Talking Nonsense: Probing Large Language Models' Understanding of Adversarial Gibberish Inputs"

49 / 49 papers shown

Title
Do we really have to filter out random noise in pre-training data for language models? Jinghan Ru Yuxin Xie Xianwei Zhuang Yuguo Yin Zhihui Guo Zhiming Liu Qianli Ren Yuexian Zou 150 4 0 10 Feb 2025
Fast Adversarial Attacks on Language Models In One GPU Minute Vinu Sankar Sadasivan Shoumik Saha Gaurang Sriramanan Priyatham Kattakinda Atoosa Malemir Chegini Soheil Feizi MIALM 73 38 0 23 Feb 2024
How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs Yi Zeng Hongpeng Lin Jingwen Zhang Diyi Yang Ruoxi Jia Weiyan Shi 72 298 0 12 Jan 2024
Tree of Attacks: Jailbreaking Black-Box LLMs Automatically Anay Mehrotra Manolis Zampetakis Paul Kassianik Blaine Nelson Hyrum Anderson Yaron Singer Amin Karbasi 71 249 0 04 Dec 2023
Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting Melanie Sclar Yejin Choi Yulia Tsvetkov Alane Suhr 81 342 0 17 Oct 2023
Jailbreaking Black Box Large Language Models in Twenty Queries Patrick Chao Alexander Robey Yan Sun Hamed Hassani George J. Pappas Eric Wong AAML 85 670 0 12 Oct 2023
AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models Xiaogeng Liu Nan Xu Muhao Chen Chaowei Xiao SILM 67 312 0 03 Oct 2023
Who's Harry Potter? Approximate Unlearning in LLMs Ronen Eldan M. Russinovich MU MoMe 142 199 0 03 Oct 2023
LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examples Jia-Yu Yao Kun-Peng Ning Zhen-Hui Liu Munan Ning Li Yuan HILM LRM AAML 59 186 0 02 Oct 2023
Open Sesame! Universal Black Box Jailbreaking of Large Language Models Raz Lapid Ron Langberg Moshe Sipper AAML 70 112 0 04 Sep 2023
Baseline Defenses for Adversarial Attacks Against Aligned Language Models Neel Jain Avi Schwarzschild Yuxin Wen Gowthami Somepalli John Kirchenbauer Ping Yeh-Chiang Micah Goldblum Aniruddha Saha Jonas Geiping Tom Goldstein AAML 103 384 0 01 Sep 2023
Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questions Pouya Pezeshkpour Estevam R. Hruschka LRM 40 141 0 22 Aug 2023
Universal and Transferable Adversarial Attacks on Aligned Language Models Andy Zou Zifan Wang Nicholas Carlini Milad Nasr J. Zico Kolter Matt Fredrikson 231 1,436 0 27 Jul 2023
Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchilla Tom Lieberum Matthew Rahtz János Kramár Neel Nanda G. Irving Rohin Shah Vladimir Mikulik 82 113 0 18 Jul 2023
Llama 2: Open Foundation and Fine-Tuned Chat Models Hugo Touvron Louis Martin Kevin R. Stone Peter Albert Amjad Almahairi ... Sharan Narang Aurelien Rodriguez Robert Stojnic Sergey Edunov Thomas Scialom AI4MH ALM 258 11,765 0 18 Jul 2023
Jailbroken: How Does LLM Safety Training Fail? Alexander Wei Nika Haghtalab Jacob Steinhardt 185 948 0 05 Jul 2023
Are aligned neural networks adversarially aligned? Nicholas Carlini Milad Nasr Christopher A. Choquette-Choo Matthew Jagielski Irena Gao ... Pang Wei Koh Daphne Ippolito Katherine Lee Florian Tramèr Ludwig Schmidt AAML 53 244 0 26 Jun 2023
DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models Wei Ping Weixin Chen Hengzhi Pei Chulin Xie Mintong Kang ... Zinan Lin Yuk-Kit Cheng Sanmi Koyejo D. Song Yue Liu 85 413 0 20 Jun 2023
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena Lianmin Zheng Wei-Lin Chiang Ying Sheng Siyuan Zhuang Zhanghao Wu ... Dacheng Li Eric Xing Haotong Zhang Joseph E. Gonzalez Ion Stoica ALM OSLM ELM 306 4,253 0 09 Jun 2023
The Internal State of an LLM Knows When It's Lying A. Azaria Tom Michael Mitchell HILM 248 329 0 26 Apr 2023
OpenAssistant Conversations -- Democratizing Large Language Model Alignment Andreas Kopf Yannic Kilcher Dimitri von Rutte Sotiris Anagnostidis Zhi Rui Tam ... Arnav Dantuluri Andrew Maguire Christoph Schuhmann Huu Nguyen A. Mattick ALM LM&MA 102 625 0 14 Apr 2023
Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attacks Daniel Kang Xuechen Li Ion Stoica Carlos Guestrin Matei A. Zaharia Tatsunori Hashimoto AAML 81 249 0 11 Feb 2023
Constitutional AI: Harmlessness from AI Feedback Yuntao Bai Saurav Kadavath Sandipan Kundu Amanda Askell John Kernion ... Dario Amodei Nicholas Joseph Sam McCandlish Tom B. Brown Jared Kaplan SyDa MoMe 162 1,603 0 15 Dec 2022
Demystifying Prompts in Language Models via Perplexity Estimation Hila Gonen Srini Iyer Terra Blevins Noah A. Smith Luke Zettlemoyer LRM 101 210 0 08 Dec 2022
Ignore Previous Prompt: Attack Techniques For Language Models Fábio Perez Ian Ribeiro SILM 78 434 0 17 Nov 2022
Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small Kevin Wang Alexandre Variengien Arthur Conmy Buck Shlegeris Jacob Steinhardt 287 549 0 01 Nov 2022
Saliency Map Verbalization: Comparing Feature Importance Representations from Model-free and Instruction-based Methods Nils Feldhus Leonhard Hennig Maximilian Dustin Nasert Christopher Ebert Robert Schwarzenberg Sebastian Möller FAtt 46 20 0 13 Oct 2022
Improving alignment of dialogue agents via targeted human judgements Amelia Glaese Nat McAleese Maja Trkebacz John Aslanides Vlad Firoiu ... John F. J. Mellor Demis Hassabis Koray Kavukcuoglu Lisa Anne Hendricks G. Irving ALM AAML 291 523 0 28 Sep 2022
In-context Learning and Induction Heads Catherine Olsson Nelson Elhage Neel Nanda Nicholas Joseph Nova Dassarma ... Tom B. Brown Jack Clark Jared Kaplan Sam McCandlish C. Olah 303 510 0 24 Sep 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 743 12,835 0 04 Mar 2022
Locating and Editing Factual Associations in GPT Kevin Meng David Bau A. Andonian Yonatan Belinkov KELM 187 1,330 0 10 Feb 2022
Where do Models go Wrong? Parameter-Space Saliency Maps for Explainability Roman Levin Manli Shu Eitan Borgnia Furong Huang Micah Goldblum Tom Goldstein FAtt AAML 33 10 0 03 Aug 2021
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity Yao Lu Max Bartolo Alastair Moore Sebastian Riedel Pontus Stenetorp AILaw LRM 347 1,177 0 18 Apr 2021
Gradient-based Adversarial Attacks against Text Transformers Chuan Guo Alexandre Sablayrolles Hervé Jégou Douwe Kiela SILM 134 242 0 15 Apr 2021
LowKey: Leveraging Adversarial Attacks to Protect Social Media Users from Facial Recognition Valeriia Cherepanova Micah Goldblum Harrison Foley Shiyuan Duan John P. Dickerson Gavin Taylor Tom Goldstein AAML PICV 49 136 0 20 Jan 2021
Fine-Tuning Language Models from Human Preferences Daniel M. Ziegler Nisan Stiennon Jeff Wu Tom B. Brown Alec Radford Dario Amodei Paul Christiano G. Irving ALM 443 1,717 0 18 Sep 2019
Universal Adversarial Triggers for Attacking and Analyzing NLP Eric Wallace Shi Feng Nikhil Kandpal Matt Gardner Sameer Singh AAML SILM 104 865 0 20 Aug 2019
What Does BERT Look At? An Analysis of BERT's Attention Kevin Clark Urvashi Khandelwal Omer Levy Christopher D. Manning MILM 200 1,592 0 11 Jun 2019
This Email Could Save Your Life: Introducing the Task of Email Subject Line Generation Rui Zhang Joel R. Tetreault 66 77 0 08 Jun 2019
Physical Adversarial Examples for Object Detectors Kevin Eykholt Ivan Evtimov Earlence Fernandes Yue Liu Amir Rahmati Florian Tramèr Atul Prakash Tadayoshi Kohno D. Song AAML 81 468 0 20 Jul 2018
Generating Natural Language Adversarial Examples M. Alzantot Yash Sharma Ahmed Elgohary Bo-Jhang Ho Mani B. Srivastava Kai-Wei Chang AAML 388 928 0 21 Apr 2018
Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifiers Ji Gao Jack Lanchantin M. Soffa Yanjun Qi AAML 127 719 0 13 Jan 2018
Towards Evaluating the Robustness of Neural Networks Nicholas Carlini D. Wagner OOD AAML 214 8,533 0 16 Aug 2016
Rationalizing Neural Predictions Tao Lei Regina Barzilay Tommi Jaakkola 99 811 0 13 Jun 2016
The Limitations of Deep Learning in Adversarial Settings Nicolas Papernot Patrick McDaniel S. Jha Matt Fredrikson Z. Berkay Celik A. Swami AAML 74 3,955 0 24 Nov 2015
Explaining and Harnessing Adversarial Examples Ian Goodfellow Jonathon Shlens Christian Szegedy AAML GAN 225 19,017 0 20 Dec 2014
Intriguing properties of neural networks Christian Szegedy Wojciech Zaremba Ilya Sutskever Joan Bruna D. Erhan Ian Goodfellow Rob Fergus AAML 227 14,893 1 21 Dec 2013
Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps Karen Simonyan Andrea Vedaldi Andrew Zisserman FAtt 244 7,279 0 20 Dec 2013
Visualizing and Understanding Convolutional Networks Matthew D. Zeiler Rob Fergus FAtt SSL 441 15,861 0 12 Nov 2013