Benchmarking Practices in LLM-driven Offensive Security: Testbeds, Metrics, and Experiment Design

14 April 2025

Papers citing "Benchmarking Practices in LLM-driven Offensive Security: Testbeds, Metrics, and Experiment Design"

9 / 9 papers shown

Title
Towards Automated Penetration Testing: Introducing LLM Benchmark, Analysis, and Improvements I. Isozaki Manil Shrestha Rick Console Edward Kim ELM 97 7 0 24 Feb 2025
Can LLMs Hack Enterprise Networks? Autonomous Assumed Breach Penetration-Testing Active Directory Networks A. Happe Jürgen Cito 88 4 0 06 Feb 2025
Teams of LLM Agents can Exploit Zero-Day Vulnerabilities Richard Fang Antony Kellermann Akul Gupta Qiusi Zhan Richard Fang R. Bindu Daniel Kang LLMAG 65 34 0 02 Jun 2024
Large Language Models for Cyber Security: A Systematic Literature Review HanXiang Xu Shenao Wang Ningke Li Kaidi Wang Yanjie Zhao Kai Chen Ting Yu Yang Liu Haoyu Wang 95 40 0 08 May 2024
CyberSecEval 2: A Wide-Ranging Cybersecurity Evaluation Suite for Large Language Models Manish P Bhatt Sahana Chennabasappa Yue Li Cyrus Nikolaidis Daniel Song ... Yaohui Chen Dhaval Kapil David Molnar Spencer Whitman Joshua Saxe ELM 76 40 0 19 Apr 2024
Review of Generative AI Methods in Cybersecurity Yagmur Yigit William J. Buchanan Madjid G Tehrani Leandros A. Maglaras AAML 115 23 0 13 Mar 2024
AutoAttacker: A Large Language Model Guided System to Implement Automatic Cyber-attacks Jiacen Xu Jack W. Stokes Geoff McDonald Xuesong Bai David Marshall Siyue Wang Adith Swaminathan Zhou Li 75 58 0 02 Mar 2024
Getting pwn'd by AI: Penetration Testing with Large Language Models A. Happe Jürgen Cito 60 81 0 24 Jul 2023
Large Language Models Michael R Douglas LLMAG LM&MA 127 623 0 11 Jul 2023