Can LLMs Separate Instructions From Data? And What Do We Even Mean By That?

Can LLMs Separate Instructions From Data? And What Do We Even Mean By That?

11 March 2024

Sahar Abdelnabi

Mario Fritz

Christoph H. Lampert

Papers citing "Can LLMs Separate Instructions From Data? And What Do We Even Mean By That?"

15 / 15 papers shown

Title
The Illusion of Role Separation: Hidden Shortcuts in LLM Role Learning (and How to Fix Them) Zihao Wang Yibo Jiang Jiahao Yu Heqing Huang 35 0 0 01 May 2025
CachePrune: Neural-Based Attribution Defense Against Indirect Prompt Injection Attacks Rui Wang Junda Wu Yu Xia Tong Yu R. Zhang Ryan Rossi Lina Yao Julian McAuley AAML SILM 51 0 0 29 Apr 2025
Exploring How LLMs Capture and Represent Domain-Specific Knowledge Mirian Hipolito Garcia Camille Couturier Daniel Madrigal Diaz Ankur Mallick Anastasios Kyrillidis Robert Sim Victor Rühle Saravan Rajmohan 30 0 0 23 Apr 2025
ASIDE: Architectural Separation of Instructions and Data in Language Models Egor Zverev Evgenii Kortukov Alexander Panfilov Soroush Tabesh Alexandra Volkova Sebastian Lapuschkin Wojciech Samek Christoph H. Lampert AAML 54 1 0 13 Mar 2025
Control Illusion: The Failure of Instruction Hierarchies in Large Language Models Yilin Geng Yiming Li Honglin Mu Xudong Han Timothy Baldwin Omri Abend Eduard H. Hovy Lea Frermann 41 2 0 21 Feb 2025
Attention Tracker: Detecting Prompt Injection Attacks in LLMs Kuo-Han Hung Ching-Yun Ko Ambrish Rawat I-Hsin Chung Winston H. Hsu Pin-Yu Chen 49 7 0 01 Nov 2024
FATH: Authentication-based Test-time Defense against Indirect Prompt Injection Attacks Jiongxiao Wang Fangzhou Wu Wendi Li Jinsheng Pan Edward Suh Zhuoqing Mao Muhao Chen Chaowei Xiao AAML 40 6 0 28 Oct 2024
Instructional Segment Embedding: Improving LLM Safety with Instruction Hierarchy Tong Wu Shujian Zhang Kaiqiang Song Silei Xu Sanqiang Zhao Ravi Agrawal Sathish Indurthi Chong Xiang Prateek Mittal Wenxuan Zhou 39 8 0 09 Oct 2024
Casper: Prompt Sanitization for Protecting User Privacy in Web-Based Large Language Models Chun Jie Chong Chenxi Hou Z. Yao S. Talebi 42 4 0 13 Aug 2024
Soft Begging: Modular and Efficient Shielding of LLMs against Prompt Injection and Jailbreaking based on Prompt Tuning Simon Ostermann Kevin Baum Christoph Endres Julia Masloh P. Schramowski AAML 48 1 0 03 Jul 2024
SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal Tinghao Xie Xiangyu Qi Yi Zeng Yangsibo Huang Udari Madhushani Sehwag ... Bo Li Kai Li Danqi Chen Peter Henderson Prateek Mittal ALM ELM 55 51 0 20 Jun 2024
AgentDojo: A Dynamic Environment to Evaluate Attacks and Defenses for LLM Agents Edoardo Debenedetti Jie Zhang Mislav Balunović Luca Beurer-Kellner Marc Fischer Florian Tramèr LLMAG AAML 50 26 1 19 Jun 2024
The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions Eric Wallace Kai Y. Xiao R. Leike Lilian Weng Johannes Heidecke Alex Beutel SILM 58 117 0 19 Apr 2024
Jatmo: Prompt Injection Defense by Task-Specific Finetuning Julien Piet Maha Alrashed Chawin Sitawarin Sizhe Chen Zeming Wei Elizabeth Sun Basel Alomair David A. Wagner AAML SyDa 75 52 0 29 Dec 2023
Privacy in Large Language Models: Attacks, Defenses and Future Directions Haoran Li Yulin Chen Jinglong Luo Yan Kang Xiaojin Zhang Qi Hu Chunkit Chan Yangqiu Song PILM 45 42 0 16 Oct 2023