大语言模型在双用途生物任务中的新手提升
LLM Novice Uplift on Dual-Use, In Silico Biology Tasks
AI总结：
研究探讨大语言模型是否能提升新手在生物任务中的表现，通过多模型多基准测试发现LLM显著提升准确性，并指出单用途LLM可能超越辅助新手的表现。
摘要：
大语言模型（LLMs）在生物基准测试中表现日益优异，但尚不清楚它们是否能提升新手用户——即使人类在没有互联网资源时的表现。这种不确定性对于理解科学加速和双用途风险至关重要。我们进行了一项多模型、多基准的人类提升研究，比较了有LLM访问权限的新手与仅互联网访问的新手在八个生物安全相关任务集中的表现。参与者在复杂问题上拥有充足的时间（最复杂的任务可达13小时）。我们发现，LLM访问提供了显著的提升：拥有LLM的新手准确性是对照组的4.16倍（95%置信区间[2.63, 6.87]）。在四个有专家基线的基准测试中，拥有LLM的新手在三个基准测试中超越了专家。令人惊讶的是，独立的LLM经常超过有LLM辅助的新手，表明用户并未从LLM中提取出最强的贡献。大多数参与者（89.6%）报告说，尽管有安全措施，获取双用途相关信息的困难很小。总体而言，LLM在生物任务上显著提升了新手的表现，这些任务以前只由受过训练的从业者完成，这强调了需要持续的、互动的提升评估，与传统基准测试相结合。
英文摘要：
Large language models (LLMs) perform increasingly well on biology benchmarks, but it remains unclear whether they uplift novice users -- i.e., enable humans to perform better than with internet-only resources. This uncertainty is central to understanding both scientific acceleration and dual-use risk. We conducted a multi-model, multi-benchmark human uplift study comparing novices with LLM access versus internet-only access across eight biosecurity-relevant task sets. Participants worked on complex problems with ample time (up to 13 hours for the most involved tasks). We found that LLM access provided substantial uplift: novices with LLMs were 4.16 times more accurate than controls (95% CI [2.63, 6.87]). On four benchmarks with available expert baselines (internet-only), novices with LLMs outperformed experts on three of them. Perhaps surprisingly, standalone LLMs often exceeded LLM-assisted novices, indicating that users were not eliciting the strongest available contributions from the LLMs. Most participants (89.6%) reported little difficulty obtaining dual-use-relevant information despite safeguards. Overall, LLMs substantially uplift novices on biological tasks previously reserved for trained practitioners, underscoring the need for sustained, interactive uplift evaluations alongside traditional benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
