arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.02297cs.CRcs.AI
收藏

ZeroDayBench: 评估LLM代理在未见的零日漏洞上的能力以应对网络防御

ZeroDayBench: Evaluating LLM Agents on Unseen Zero-Day Vulnerabilities for Cyberdefense

Nancy Lau, Louis Sloot, Jyoutir Raj, Giuseppe Marco Boscardin, Evan Harris, Dylan Bowman, Mario Brajkovski, Jaideep Chawla, Dan Zhao

AI总结:

ZeroDayBench评估了前沿LLM在未见零日漏洞上的自主修复能力,揭示了其在主动网络防御中的局限性及改进方向。

摘要:

大型语言模型(LLM)正越来越多地被部署为软件工程代理,以自主地贡献于代码库。这些代理的一个主要优势是它们能够找到并修补它们所监督的代码库中的安全漏洞。为了估计这些代理在该领域的能力,我们引入了ZeroDayBench基准测试,其中LLM代理在开源代码库中发现并修补22个新的关键漏洞。我们专注于三种流行的前沿代理LLM:GPT-5.2、Claude Sonnet 4.5和Grok 4.1。我们发现前沿LLM目前尚无法自主解决我们的任务,并观察到一些行为模式,表明这些模型在主动网络防御领域可以如何改进。

英文摘要:

Large language models (LLMs) are increasingly being deployed as software engineering agents that autonomously contribute to repositories. A major benefit these agents present is their ability to find and patch security vulnerabilities in the codebases they oversee. To estimate the capability of agents in this domain, we introduce ZeroDayBench, a benchmark where LLM agents find and patch 22 novel critical vulnerabilities in open-source codebases. We focus our efforts on three popular frontier agentic LLMs: GPT-5.2, Claude Sonnet 4.5, and Grok 4.1. We find that frontier LLMs are not yet capable of autonomously solving our tasks and observe some behavioral patterns that suggest how these models can be improved in the domain of proactive cyberdefense.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑