Can LLMs Follow Simple Rules?

6 November 2023

Papers citing "Can LLMs Follow Simple Rules?"

28 / 28 papers shown

Title
GuideBench: Benchmarking Domain-Oriented Guideline Following for LLM Agents Lingxiao Diao Xinyue Xu Wanxuan Sun Cheng Yang Zhuosheng Zhang LLMAG ALM ELM 14 0 0 16 May 2025
WALL-E 2.0: World Alignment by NeuroSymbolic Learning improves World Model-based LLM Agents Siyu Zhou Tianyi Zhou Yijun Yang Guodong Long Deheng Ye Jing Jiang Chengqi Zhang LM&Ro 32 0 0 22 Apr 2025
ASIDE: Architectural Separation of Instructions and Data in Language Models Egor Zverev Evgenii Kortukov Alexander Panfilov Soroush Tabesh Alexandra Volkova Sebastian Lapuschkin Wojciech Samek Christoph H. Lampert AAML 54 1 0 13 Mar 2025
Scaling Trends in Language Model Robustness Nikolhaus Howe Michal Zajac I. R. McKenzie Oskar Hollinsworth Tom Tseng Aaron David Tucker Pierre-Luc Bacon Adam Gleave 117 2 0 21 Feb 2025
Divide-Verify-Refine: Can LLMs Self-Align with Complex Instructions? Xianren Zhang Xianfeng Tang Hui Liu Zongyu Wu Qi He Dongwon Lee Suhang Wang ALM 41 0 0 16 Oct 2024
COMPL-AI Framework: A Technical Interpretation and LLM Benchmarking Suite for the EU Artificial Intelligence Act Philipp Guldimann Alexander Spiridonov Robin Staab Nikola Jovanović Mark Vero ... Mislav Balunović Nikola Konstantinov Pavol Bielik Petar Tsankov Martin Vechev ELM 53 4 0 10 Oct 2024
WALL-E: World Alignment by Rule Learning Improves World Model-based LLM Agents Siyu Zhou Tianyi Zhou Yijun Yang Guodong Long Deheng Ye Jing Jiang Chengqi Zhang LLMAG LM&Ro 28 3 0 09 Oct 2024
LLM Self-Correction with DeCRIM: Decompose, Critique, and Refine for Enhanced Following of Instructions with Multiple Constraints Thomas Palmeira Ferraz Kartik Mehta Yu-Hsiang Lin Haw-Shiuan Chang Shereen Oraby Sijia Liu Vivek Subramanian Tagyoung Chung Mohit Bansal Nanyun Peng 56 8 0 09 Oct 2024
Instructional Segment Embedding: Improving LLM Safety with Instruction Hierarchy Tong Wu Shujian Zhang Kaiqiang Song Silei Xu Sanqiang Zhao Ravi Agrawal Sathish Indurthi Chong Xiang Prateek Mittal Wenxuan Zhou 45 8 0 09 Oct 2024
SysBench: Can Large Language Models Follow System Messages? Yanzhao Qin Tao Zhang Tao Zhang Yanjun Shen Wenjing Luo ... Yujing Qiao Weipeng Chen Zenan Zhou Wentao Zhang Bin Cui ALM 113 7 0 20 Aug 2024
Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress? Richard Ren Steven Basart Adam Khoja Alice Gatti Long Phan ... Alexander Pan Gabriel Mukobi Ryan H. Kim Stephen Fitz Dan Hendrycks ELM 26 21 0 31 Jul 2024
Beyond Instruction Following: Evaluating Inferential Rule Following of Large Language Models Wangtao Sun Chenxiang Zhang Xueyou Zhang Ziyang Huang Haotian Xu Pei Chen Shizhu He Jun Zhao Kang Liu ELM LRM 35 0 0 11 Jul 2024
The SIFo Benchmark: Investigating the Sequential Instruction Following Ability of Large Language Models Xinyi Chen Baohao Liao Jirui Qi Panagiotis Eustratiadis Christof Monz Arianna Bisazza Maarten de Rijke ALM ELM LRM 33 5 0 28 Jun 2024
Model Internals-based Answer Attribution for Trustworthy Retrieval-Augmented Generation Jirui Qi Gabriele Sarti Raquel Fernández Arianna Bisazza RALM 45 6 0 19 Jun 2024
Human-AI Safety: A Descendant of Generative AI and Control Systems Safety Andrea V. Bajcsy J. F. Fisac 40 7 0 16 May 2024
From Complex to Simple: Enhancing Multi-Constraint Complex Instruction Following Ability of Large Language Models Qi He Jie Zeng Qianxi He Jiaqing Liang Yanghua Xiao 32 10 0 24 Apr 2024
CantTalkAboutThis: Aligning Language Models to Stay on Topic in Dialogues Makesh Narsimhan Sreedhar Traian Rebedea Shaona Ghosh Jiaqi Zeng Christopher Parisien ALM 35 4 0 04 Apr 2024
SoFA: Shielded On-the-fly Alignment via Priority Rule Following Xinyu Lu Bowen Yu Yaojie Lu Hongyu Lin Haiyang Yu Le Sun Xianpei Han Yongbin Li 63 13 0 27 Feb 2024
Generative AI Security: Challenges and Countermeasures Banghua Zhu Norman Mu Jiantao Jiao David Wagner AAML SILM 61 8 0 20 Feb 2024
Can LLMs Reason with Rules? Logic Scaffolding for Stress-Testing and Improving LLMs Siyuan Wang Zhongyu Wei Yejin Choi Xiang Ren ReLM ELM LRM 16 20 0 18 Feb 2024
Feedback Loops With Language Models Drive In-Context Reward Hacking Alexander Pan Erik Jones Meena Jagadeesan Jacob Steinhardt KELM 53 26 0 09 Feb 2024
Red-Teaming for Generative AI: Silver Bullet or Security Theater? Michael Feffer Anusha Sinha Wesley Hanwen Deng Zachary Chase Lipton Hoda Heidari AAML 38 67 0 29 Jan 2024
Tensor Trust: Interpretable Prompt Injection Attacks from an Online Game Sam Toyer Olivia Watkins Ethan Mendes Justin Svegliato Luke Bailey ... Karim Elmaaroufi Pieter Abbeel Trevor Darrell Alan Ritter Stuart J. Russell 21 71 0 02 Nov 2023
LLM Platform Security: Applying a Systematic Evaluation Framework to OpenAI's ChatGPT Plugins Umar Iqbal Tadayoshi Kohno Franziska Roesner ELM SILM 74 48 0 19 Sep 2023
Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned Deep Ganguli Liane Lovitt John Kernion Amanda Askell Yuntao Bai ... Nicholas Joseph Sam McCandlish C. Olah Jared Kaplan Jack Clark 231 446 0 23 Aug 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 339 12,003 0 04 Mar 2022
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown D. Song Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 290 1,824 0 14 Dec 2020
Fine-Tuning Language Models from Human Preferences Daniel M. Ziegler Nisan Stiennon Jeff Wu Tom B. Brown Alec Radford Dario Amodei Paul Christiano G. Irving ALM 298 1,610 0 18 Sep 2019