329

Agents of Chaos

Natalie Shapira
Chris Wendler
Avery Yen
Gabriele Sarti
Koyena Pal
Olivia Floody
Adam Belfki
Alex Loftus
Aditya Ratan Jannali
Nikhil Prakash
Jasmine Cui
Giordano Rogers
Jannik Brinkmann
Can Rager
Amir Zur
Michael Ripa
Aruna Sankaranarayanan
David Atkinson
Rohit Gandikota
Jaden Fiotto-Kaufman
EunJeong Hwang
Hadas Orgad
P Sam Sahil
Negev Taglicht
Tomer Shabtay
Atai Ambus
Nitay Alon
Shiri Oron
Ayelet Gordon-Tapiero
Yotam Kaplan
Vered Shwartz
Tamar Rott Shaham
Christoph Riedl
Reuth Mirsky
Maarten Sap
David Manheim
Tomer Ullman
David Bau
Main:47 Pages
47 Figures
Bibliography:11 Pages
Appendix:26 Pages
Abstract

We report an exploratory red-teaming study of autonomous language-model-powered agents deployed in a live laboratory environment with persistent memory, email accounts, Discord access, file systems, and shell execution. Over a two-week period, twenty AI researchers interacted with the agents under benign and adversarial conditions. Focusing on failures emerging from the integration of language models with autonomy, tool use, and multi-party communication, we document eleven representative case studies. Observed behaviors include unauthorized compliance with non-owners, disclosure of sensitive information, execution of destructive system-level actions, denial-of-service conditions, uncontrolled resource consumption, identity spoofing vulnerabilities, cross-agent propagation of unsafe practices, and partial system takeover. In several cases, agents reported task completion while the underlying system state contradicted those reports. We also report on some of the failed attempts. Our findings establish the existence of security-, privacy-, and governance-relevant vulnerabilities in realistic deployment settings. These behaviors raise unresolved questions regarding accountability, delegated authority, and responsibility for downstream harms, and warrant urgent attention from legal scholars, policymakers, and researchers across disciplines. This report serves as an initial empirical contribution to that broader conversation.

View on arXiv
Comments on this paper