arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2512.08451cs.LGcs.AIcs.CYcs.ET
收藏

生物威胁基准生成框架用于评估前沿AI模型II:基准生成过程

Biothreat Benchmark Generation Framework for Evaluating Frontier AI Models II: Benchmark Generation Process

Nemesys Insights, LLC(Nemesys洞察有限责任公司);Frontier Design Group, LLC(前沿设计集团有限责任公司)

Gary Ackerman, Zachary Kallenborn, Anna Wetzel, Hayley Peterson, Jenna LaTourette, Olivia Shoemaker, Brandon Behlendorf, Sheriff Almakki, Doug Clifford, Noah Sheinbaum

AI总结:

本文提出生物威胁基准生成框架,通过三种方法生成细菌生物威胁基准数据集,用于评估前沿AI模型的生物安全风险。

摘要:

快速发展的前沿人工智能(AI)模型,尤其是大型语言模型(LLMs),可能被用于生物恐怖主义或获取生物武器的可能性,引发了显著的政策、学术和公众关注。模型开发者和政策制定者都希望量化和减轻这些风险,其中一个重要组成部分是开发能够评估特定模型生物安全风险的基准。本文是系列三篇论文中的第二篇,描述了新型生物威胁基准生成(BBG)框架的第二部分:生成细菌生物威胁基准(B3)数据集。开发过程涉及三种互补方法:1)基于网络的提示生成,2)红队行动,3)挖掘现有基准语料库,以生成超过7000个与在项目第一部分开发的Task-Query架构相关的潜在基准。经过去重、提升诊断性评估和一般质量控制措施后,候选者被缩减为1010个最终基准。该过程确保这些基准在提供提升方面具有诊断性;直接相关于生物安全威胁;并且与更大的生物安全架构对齐,允许在不同分析层次上进行细致分析。

英文摘要:

The potential for rapidly-evolving frontier artificial intelligence (AI) models, especially large language models (LLMs), to facilitate bioterrorism or access to biological weapons has generated significant policy, academic, and public concern. Both model developers and policymakers seek to quantify and mitigate any risk, with an important element of such efforts being the development of model benchmarks that can assess the biosecurity risk posed by a particular model. This paper, the second in a series of three, describes the second component of a novel Biothreat Benchmark Generation (BBG) framework: the generation of the Bacterial Biothreat Benchmark (B3) dataset. The development process involved three complementary approaches: 1) web-based prompt generation, 2) red teaming, and 3) mining existing benchmark corpora, to generate over 7,000 potential benchmarks linked to the Task-Query Architecture that was developed during the first component of the project. A process of de-duplication, followed by an assessment of uplift diagnosticity, and general quality control measures, reduced the candidates to a set of 1,010 final benchmarks. This procedure ensured that these benchmarks are a) diagnostic in terms of providing uplift; b) directly relevant to biosecurity threats; and c) are aligned with a larger biosecurity architecture permitting nuanced analysis at different levels of analysis.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑