EnvInjection: Environmental Prompt Injection Attack to Multi-modal Web Agents

EnvInjection: Environmental Prompt Injection Attack to Multi-modal Web Agents

16 May 2025

Neil Zhenqiang Gong

ArXiv (abs)PDF HTML

Papers citing "EnvInjection: Environmental Prompt Injection Attack to Multi-modal Web Agents"

18 / 18 papers shown

Title
Transferable Adversarial Attacks on Black-Box Vision-Language Models Kai Hu Weichen Yu Lefei Zhang Alexander Robey Andy Zou Chengming Xu Haoqi Hu Matt Fredrikson AAML VLM 125 2 0 02 May 2025
On the Robustness of GUI Grounding Models Against Image Attacks Haoren Zhao Tianyi Chen Zhen Wang AAML 120 3 0 07 Apr 2025
Gemma 3 Technical Report Gemma Team Aishwarya B Kamath Johan Ferret Shreya Pathak Nino Vieillard ... Harshal Tushar Lehri Hussein Hazimeh Ian Ballantyne Idan Szpektor Ivan Nardini VLM 191 136 0 25 Mar 2025
Attacking Multimodal OS Agents with Malicious Image Patches Lukas Aichberger Alasdair Paren Y. Gal Philip Torr Adel Bibi AAML 113 5 0 13 Mar 2025
Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs Abdelrahman Abouelenin Atabak Ashfaq Adam Atkinson Hany Awadalla Nguyen Bach ... Ishmam Zabir Yunan Zhang Li Zhang Yanzhe Zhang Xiren Zhou MoE SyDa 120 70 0 03 Mar 2025
Qwen2.5-VL Technical Report S. Bai Keqin Chen Xuejing Liu Jialin Wang Wenbin Ge ... Zesen Cheng Hang Zhang Zhibo Yang Haiyang Xu Junyang Lin VLM 384 699 0 20 Feb 2025
UI-TARS: Pioneering Automated GUI Interaction with Native Agents Yujia Qin Yining Ye Junjie Fang Han Wang Shihao Liang ... Haifeng Liu F. Lin Tao Peng Xin Liu Guang Shi LLMAG LM&Ro 102 69 0 21 Jan 2025
Attacking Vision-Language Computer Agents via Pop-ups Yanzhe Zhang Tao Yu Diyi Yang AAML VLM 136 34 0 04 Nov 2024
AgentDojo: A Dynamic Environment to Evaluate Attacks and Defenses for LLM Agents Edoardo Debenedetti Jie Zhang Mislav Balunović Luca Beurer-Kellner Marc Fischer Florian Tramèr LLMAG AAML 123 45 1 19 Jun 2024
An Image Is Worth 1000 Lies: Adversarial Transferability across Prompts on Vision-Language Models Haochen Luo Jindong Gu Fengyuan Liu Philip Torr VLM VPVLM AAML 84 24 0 14 Mar 2024
WIPI: A New Web Threat for LLM-Driven Web Agents Fangzhou Wu Shutong Wu Yulong Cao Chaowei Xiao LLMAG 82 28 0 26 Feb 2024
VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks Jing Yu Koh Robert Lo Lawrence Jang Vikram Duvvur Ming Chong Lim Po-Yu Huang Graham Neubig Shuyan Zhou Ruslan Salakhutdinov Daniel Fried 113 0 0 24 Jan 2024
GPT-4V(ision) is a Generalist Web Agent, if Grounded Boyuan Zheng Boyu Gou Jihyung Kil Huan Sun Yu-Chuan Su MLLM VLM LLMAG 130 264 0 03 Jan 2024
Formalizing and Benchmarking Prompt Injection Attacks and Defenses Yupei Liu Yuqi Jia Runpeng Geng Jinyuan Jia Neil Zhenqiang Gong SILM LLMAG 120 95 0 19 Oct 2023
Visual Adversarial Examples Jailbreak Aligned Large Language Models Xiangyu Qi Kaixuan Huang Ashwinee Panda Peter Henderson Mengdi Wang Prateek Mittal AAML 107 171 0 22 Jun 2023
Towards Deep Learning Models Resistant to Adversarial Attacks Aleksander Madry Aleksandar Makelov Ludwig Schmidt Dimitris Tsipras Adrian Vladu SILM OOD 325 12,151 0 19 Jun 2017
Towards Evaluating the Robustness of Neural Networks Nicholas Carlini D. Wagner OOD AAML 284 8,593 0 16 Aug 2016
U-Net: Convolutional Networks for Biomedical Image Segmentation Olaf Ronneberger Philipp Fischer Thomas Brox SSeg 3DV 1.9K 77,520 0 18 May 2015