InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents

5 March 2024

Papers citing "InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents"

50 / 56 papers shown

Title
Is Your Prompt Safe? Investigating Prompt Injection Attacks Against Open-Source LLMs Jiawen Wang Pritha Gupta Ivan Habernal Eyke Hüllermeier SILM AAML 29 0 0 20 May 2025
ToolSpectrum : Towards Personalized Tool Utilization for Large Language Models Zihao Cheng Hongru Wang Zeming Liu Yuhang Guo Yuanfang Guo Yunhong Wang Haifeng Wang 12 0 0 19 May 2025
A Survey on the Safety and Security Threats of Computer-Using Agents: JARVIS or Ultron? Ada Chen Yongjiang Wu Jingyang Zhang Shu Yang Jen-tse Huang Kun Wang Wenxuan Wang Shuai Wang ELM 17 0 0 16 May 2025
Practical Reasoning Interruption Attacks on Reasoning Large Language Models Yu Cui Cong Zuo SILM AAML LRM 34 0 0 10 May 2025
System Prompt Poisoning: Persistent Attacks on Large Language Models Beyond User Injection Jiawei Guo Haipeng Cai SILM AAML 29 0 0 10 May 2025
AGENTFUZZER: Generic Black-Box Fuzzing for Indirect Prompt Injection against LLM Agents Zhun Wang Vincent Siu Zhe Ye Tianneng Shi Yuzhou Nie Xuandong Zhao Chenguang Wang Wenbo Guo Dawn Song LLMAG AAML 38 0 0 09 May 2025
Defending against Indirect Prompt Injection by Instruction Detection Tongyu Wen Chenglong Wang Xiyuan Yang Haoyu Tang Yueqi Xie Lingjuan Lyu Zhicheng Dou Fangzhao Wu AAML 34 0 0 08 May 2025
Safeguard-by-Development: A Privacy-Enhanced Development Paradigm for Multi-Agent Collaboration Systems Jian Cui Zichuan Li Luyi Xing Xiaojing Liao 29 0 0 07 May 2025
ACE: A Security Architecture for LLM-Integrated App Systems Evan Li Tushin Mallick Evan Rose William K. Robertson Alina Oprea Cristina Nita-Rotaru 52 0 0 29 Apr 2025
CachePrune: Neural-Based Attribution Defense Against Indirect Prompt Injection Attacks Rui Wang Junda Wu Yu Xia Tong Yu R. Zhang Ryan Rossi Lina Yao Julian McAuley AAML SILM 51 0 0 29 Apr 2025
Robustness via Referencing: Defending against Prompt Injection Attacks by Referencing the Executed Instruction Yuxiao Chen Haoran Li Yuan Sui Yi Liu Yufei He Yangqiu Song Bryan Hooi AAML SILM 63 0 0 29 Apr 2025
Prompt Injection Attack to Tool Selection in LLM Agents Jiawen Shi Zenghui Yuan Guiyao Tie Pan Zhou Neil Zhenqiang Gong Lichao Sun LLMAG 51 0 0 28 Apr 2025
WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks Ivan Evtimov Arman Zharmagambetov Aaron Grattafiori Chuan Guo Kamalika Chaudhuri AAML 35 1 0 22 Apr 2025
Manipulating Multimodal Agents via Cross-Modal Prompt Injection Le Wang Zonghao Ying Tianyuan Zhang Siyuan Liang Shengshan Hu Mingchuan Zhang A. Liu Xianglong Liu AAML 33 1 0 19 Apr 2025
Exploiting Fine-Grained Skip Behaviors for Micro-Video Recommendation Sanghyuck Lee Sangkeun Park Jaesung Lee 53 0 0 04 Apr 2025
Emerging Cyber Attack Risks of Medical AI Agents Jianing Qiu Lin Li Jiankai Sun Hao Wei Zhe Xu K. Lam Wu Yuan AAML 33 2 0 02 Apr 2025
Towards Trustworthy GUI Agents: A Survey Yucheng Shi Wenhao Yu Wenlin Yao Wenhu Chen Ninghao Liu 47 4 0 30 Mar 2025
sudo rm -rf agentic_security Sejin Lee Jian Kim Haon Park Ashkan Yousefpour Sangyoon Yu Min Song AAML 235 0 0 26 Mar 2025
Towards Agentic Recommender Systems in the Era of Multimodal Large Language Models Chengkai Huang Junda Wu Yu Xia Zixu Yu Ruhan Wang ... Ryan Rossi B. Kveton Dongruo Zhou Julian McAuley Lina Yao 38 1 0 20 Mar 2025
SPADE: Systematic Prompt Framework for Automated Dialogue Expansion in Machine-Generated Text Detection Haoyi Li Angela Yifei Yuan Soyeon Caren Han Christopher Leckie 52 0 0 19 Mar 2025
Multi-Agent Systems Execute Arbitrary Malicious Code Harold Triedman Rishi Jha Vitaly Shmatikov LLMAG AAML 99 2 0 15 Mar 2025
SafeArena: Evaluating the Safety of Autonomous Web Agents Ada Defne Tur Nicholas Meade Xing Han Lù Alejandra Zambrano Arkil Patel Esin Durmus Spandana Gella Karolina Stañczak Siva Reddy LLMAG ELM 87 4 0 06 Mar 2025
SpiritSight Agent: Advanced GUI Agent with One Look Zhiyuan Huang Ziming Cheng Junting Pan Zhaohui Hou Mingjie Zhan LLMAG 101 2 0 05 Mar 2025
UDora: A Unified Red Teaming Framework against LLM Agents by Dynamically Hijacking Their Own Reasoning J.N. Zhang Shuang Yang B. Li AAML LLMAG 58 0 0 28 Feb 2025
Can Indirect Prompt Injection Attacks Be Detected and Removed? Yulin Chen Haoran Li Yuan Sui Yufei He Yue Liu Yangqiu Song Bryan Hooi AAML 44 3 0 23 Feb 2025
RTBAS: Defending LLM Agents Against Prompt Injection and Privacy Leakage Peter Yong Zhong Siyuan Chen Ruiqi Wang McKenna McCall Ben L. Titzer Heather Miller Phillip B. Gibbons LLMAG 93 3 0 17 Feb 2025
Peering Behind the Shield: Guardrail Identification in Large Language Models Ziqing Yang Yixin Wu Rui Wen Michael Backes Yang Zhang 63 1 0 03 Feb 2025
The Task Shield: Enforcing Task Alignment to Defend Against Indirect Prompt Injection in LLM Agents Feiran Jia Tong Wu Xin Qin Anna Squicciarini LLMAG AAML 98 4 0 21 Dec 2024
Towards Action Hijacking of Large Language Model-based Agent Yuyang Zhang Kangjie Chen Xudong Jiang Yuxiang Sun Run Wang Lina Wang LLMAG AAML 73 3 0 14 Dec 2024
Defense Against Prompt Injection Attack by Leveraging Attack Techniques Yulin Chen Haoran Li Zihao Zheng Yangqiu Song Dekai Wu Bryan Hooi SILM AAML 50 4 0 01 Nov 2024
InjecGuard: Benchmarking and Mitigating Over-defense in Prompt Injection Guardrail Models Yiming Li Xiaogeng Liu SILM 44 5 0 30 Oct 2024
FATH: Authentication-based Test-time Defense against Indirect Prompt Injection Attacks Jiongxiao Wang Fangzhou Wu Wendi Li Jinsheng Pan Edward Suh Zhuoqing Mao Muhao Chen Chaowei Xiao AAML 40 6 0 28 Oct 2024
MobileSafetyBench: Evaluating Safety of Autonomous Agents in Mobile Device Control Juyong Lee Dongyoon Hahm June Suk Choi W. Bradley Knox Kimin Lee LLMAG ELM AAML LM&Ro 45 3 0 23 Oct 2024
Breaking ReAct Agents: Foot-in-the-Door Attack Will Get You In Itay Nakash George Kour Guy Uziel Ateret Anaby-Tavor AAML LLMAG 40 4 0 22 Oct 2024
DAWN: Designing Distributed Agents in a Worldwide Network Zahra Aminiranjbar Jianan Tang Qiudan Wang Shubha Pant Mahesh Viswanathan LLMAG AI4CE 31 2 0 11 Oct 2024
Instructional Segment Embedding: Improving LLM Safety with Instruction Hierarchy Tong Wu Shujian Zhang Kaiqiang Song Silei Xu Sanqiang Zhao Ravi Agrawal Sathish Indurthi Chong Xiang Prateek Mittal Wenxuan Zhou 45 9 0 09 Oct 2024
Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents Hanrong Zhang Jingyuan Huang Kai Mei Yifei Yao Zhenting Wang Chenlu Zhan Hongwei Wang Yongfeng Zhang AAML LLMAG ELM 51 22 0 03 Oct 2024
System-Level Defense against Indirect Prompt Injection Attacks: An Information Flow Control Perspective Fangzhou Wu Ethan Cecchetti Chaowei Xiao 41 13 0 27 Sep 2024
Breaking Agents: Compromising Autonomous LLM Agents Through Malfunction Amplification Boyang Zhang Yicong Tan Yun Shen Ahmed Salem Michael Backes Savvas Zannettou Yang Zhang LLMAG AAML 48 15 0 30 Jul 2024
The Emerged Security and Privacy of LLM Agent: A Survey with Case Studies Feng He Tianqing Zhu Dayong Ye Bo Liu Wanlei Zhou Philip S. Yu PILM LLMAG ELM 72 24 0 28 Jul 2024
Systematic Categorization, Construction and Evaluation of New Attacks against Multi-modal Mobile GUI Agents Yulong Yang Xinshan Yang Shuaidong Li Chenhao Lin Zhengyu Zhao Chao Shen Tianwei Zhang 48 1 0 12 Jul 2024
Soft Begging: Modular and Efficient Shielding of LLMs against Prompt Injection and Jailbreaking based on Prompt Tuning Simon Ostermann Kevin Baum Christoph Endres Julia Masloh P. Schramowski AAML 54 1 0 03 Jul 2024
AgentDojo: A Dynamic Environment to Evaluate Attacks and Defenses for LLM Agents Edoardo Debenedetti Jie Zhang Mislav Balunović Luca Beurer-Kellner Marc Fischer Florian Tramèr LLMAG AAML 59 28 1 19 Jun 2024
IDs for AI Systems Alan Chan Noam Kolt Peter Wills Usman Anwar Christian Schroeder de Witt Nitarshan Rajkumar Lewis Hammond David M. Krueger Lennart Heim Markus Anderljung 49 7 0 17 Jun 2024
Ranking Manipulation for Conversational Search Engines Samuel Pfrommer Yatong Bai Tanmay Gautam Somayeh Sojoudi SILM 47 4 0 05 Jun 2024
AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathways Zehang Deng Yongjian Guo Changzhou Han Wanlun Ma Junwu Xiong Sheng Wen Yang Xiang 52 26 0 04 Jun 2024
A Survey of Useful LLM Evaluation Ji-Lun Peng Sijia Cheng Egil Diau Yung-Yu Shih Po-Heng Chen Yen-Ting Lin Yun-Nung Chen LLMAG ELM 34 12 0 03 Jun 2024
Teams of LLM Agents can Exploit Zero-Day Vulnerabilities Richard Fang Antony Kellermann Akul Gupta Qiusi Zhan Richard Fang R. Bindu Daniel Kang LLMAG 40 30 0 02 Jun 2024
Tool Learning with Large Language Models: A Survey Changle Qu Sunhao Dai Xiaochi Wei Hengyi Cai Shuaiqiang Wang Dawei Yin Jun Xu Jirong Wen LLMAG 34 83 0 28 May 2024
Lockpicking LLMs: A Logit-Based Jailbreak Using Token-level Manipulation Yuxi Li Yi Liu Yuekang Li Ling Shi Gelei Deng Shengquan Chen Kailong Wang 48 12 0 20 May 2024