RRTL: Red Teaming Reasoning Large Language Models in Tool Learning

RRTL: Red Teaming Reasoning Large Language Models in Tool Learning

21 May 2025

Author Contacts:

liuyifei@bit.edu.cn bchainzhang@aliyun.com

ArXiv (abs)PDF HTML

Papers citing "RRTL: Red Teaming Reasoning Large Language Models in Tool Learning"

15 / 15 papers shown

Title
Reasoning Models Don't Always Say What They Think Yanda Chen Joe Benton Ansh Radhakrishnan Jonathan Uesato Carson E. Denison ... Vlad Mikulik Samuel R. Bowman Jan Leike Jared Kaplan E. Perez ReLM LRM 133 39 1 08 May 2025
When2Call: When (not) to Call Tools Hayley Ross Ameya Sunil Mahabaleshwarkar Yoshi Suhara 132 1 0 26 Apr 2025
ShadowCoT: Cognitive Hijacking for Stealthy Reasoning Backdoors in LLMs Gejian Zhao Hanzhou Wu Xinpeng Zhang Athanasios V. Vasilakos LRM 67 3 0 08 Apr 2025
Select Me! When You Need a Tool: A Black-box Text Attack on Tool Selection Liuji Chen Hao Gao Jinghao Zhang Qiang Liu Shu Wu Liang Wang AAML 50 1 0 07 Apr 2025
Towards Understanding the Safety Boundaries of DeepSeek Models: Evaluation and Findings Zonghao Ying Guangyi Zheng Yongxin Huang Deyue Zhang Wenxin Zhang Quanchen Zou Aishan Liu Xianglong Liu Dacheng Tao ELM 121 12 0 19 Mar 2025
DIMSUM: Discourse in Mathematical Reasoning as a Supervision Module Krish Sharma Niyar R. Barman Nicholas M. Asher Akshay Chaturvedi LRM AIMat 100 14 0 06 Mar 2025
Demonstrating specification gaming in reasoning models Alexander Bondarenko Denis Volk Dmitrii Volkov Jeffrey Ladish LRM LLMAG 60 7 0 18 Feb 2025
From Exploration to Mastery: Enabling LLMs to Master Tools via Self-Driven Interactions Changle Qu Sunhao Dai Xiaochi Wei Hengyi Cai Shuaiqiang Wang Dawei Yin Jun Xu Ji-Rong Wen 122 12 0 10 Oct 2024
ToolSword: Unveiling Safety Issues of Large Language Models in Tool Learning Across Three Stages Junjie Ye Sixian Li Guanyu Li Caishuang Huang Songyang Gao Yilong Wu Qi Zhang Tao Gui Xuanjing Huang LLMAG 131 21 0 16 Feb 2024
RoTBench: A Multi-Level Benchmark for Evaluating the Robustness of Large Language Models in Tool Learning Junjie Ye Yilong Wu Songyang Gao Caishuang Huang Sixian Li Guanyu Li Xiaoran Fan Qi Zhang Tao Gui Xuanjing Huang AAML 60 18 0 16 Jan 2024
CritiqueLLM: Towards an Informative Critique Generation Model for Evaluation of Large Language Model Generation Pei Ke Bosi Wen Andrew Feng Xiao-Yang Liu Xuanyu Lei ... Aohan Zeng Yuxiao Dong Hongning Wang Jie Tang Minlie Huang ELM ALM 99 31 0 30 Nov 2023
MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use Yue Huang Jiawen Shi Yuan Li Chenrui Fan Siyuan Wu ... Yixin Liu Pan Zhou Yao Wan Neil Zhenqiang Gong Lichao Sun LLMAG 72 92 0 04 Oct 2023
ToolQA: A Dataset for LLM Question Answering with External Tools Yuchen Zhuang Yue Yu Kuan-Chieh Wang Haotian Sun Chao Zhang ELM LLMAG 70 241 0 23 Jun 2023
Gorilla: Large Language Model Connected with Massive APIs Shishir G. Patil Tianjun Zhang Xin Wang Joseph E. Gonzalez ELM CLL ALM SyDa 84 556 0 24 May 2023
Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study Yi Liu Gelei Deng Zhengzi Xu Yuekang Li Yaowen Zheng Ying Zhang Lida Zhao Tianwei Zhang Kailong Wang Yang Liu 77 467 0 23 May 2023