arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.01605cs.CRcs.LG
收藏

CYBERSECEVAL 3:推进大型语言模型网络安全风险与能力的评估

CYBERSECEVAL 3: Advancing the Evaluation of Cybersecurity Risks and Capabilities in Large Language Models

Shengye Wan, Cyrus Nikolaidis, Daniel Song, David Molnar, James Crnkovich, Jayson Grace, Manish Bhatt, Sahana Chennabasappa, Spencer Whitman, Stephanie Ding, Vl… 展开作者

Shengye Wan, Cyrus Nikolaidis, Daniel Song, David Molnar, James Crnkovich, Jayson Grace, Manish Bhatt, Sahana Chennabasappa, Spencer Whitman, Stephanie Ding, Vlad Ionescu, Yue Li, Joshua Saxe

AI总结:

本文发布LLM网络安全评估基准CYBERSECEVAL 3,覆盖8类风险并新增三项攻击性安全能力测试,基于Llama 3及同期前沿模型比较有无缓解措施时的风险。

摘要:

我们正在发布一套面向LLM的新型安全基准套件CYBERSECEVAL 3,以继续推进关于实证衡量LLM网络安全风险与能力的讨论。CYBERSECEVAL 3评估了两大类下的8种不同风险:对第三方的风险,以及对应用程序开发者和最终用户的风险。与以往工作相比,我们新增了聚焦攻击性安全能力的领域:自动化社会工程、规模化人工攻击性网络行动,以及自主攻击性网络行动。在本文中,我们讨论了将这些基准应用于Llama 3模型和一批同期最先进LLM,使我们能够在已部署和未部署缓解措施的情况下对风险进行情境化分析。

英文摘要:

We are releasing a new suite of security benchmarks for LLMs, CYBERSECEVAL 3, to continue the conversation on empirically measuring LLM cybersecurity risks and capabilities. CYBERSECEVAL 3 assesses 8 different risks across two broad categories: risk to third parties, and risk to application developers and end users. Compared to previous work, we add new areas focused on offensive security capabilities: automated social engineering, scaling manual offensive cyber operations, and autonomous offensive cyber operations. In this paper we discuss applying these benchmarks to the Llama 3 models and a suite of contemporaneous state-of-the-art LLMs, enabling us to contextualize risks both with and without mitigations in place.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑