Factor(U,T): 通过监控其计划来控制不可信的AI
Factor(U,T): Controlling Untrusted AI by Monitoring their Plans
AI总结:
Factor(U,T)通过监控AI分解计划来控制不可信AI,实验表明在仅观察自然语言指令时,监控恶意活动效果有限,而结合子任务监控则能实现高区分度和安全性。
摘要:
随着AI能力的提升,我们越来越多地依赖强大模型来分解复杂任务——但若分解者本身是恶意的呢?因子认知协议将复杂任务分解为更简单的子任务:一个模型创建分解,而其他模型则孤立地实现子任务。先前的工作使用可信(较弱但可靠)的模型进行分解,这限制了在分解本身具有挑战性的任务中的实用性。我们引入Factor(U,T),其中不可信(更强但可能恶意)的模型进行分解,而可信模型执行子任务。监控在仅观察自然语言任务指令而非完整解决方案的情况下能否检测恶意活动?我们对BigCodeBench数据集上的控制评估中基线和红队Factor(U,T)。监控区分恶意与诚实分解表现不佳(AUROC 0.52),相比监控完整Python解决方案(AUROC 0.96)。此外,Factor(D,U),使用可信分解器和监控具体子解决方案,实现了优秀的区分(AUROC 0.96)和强安全性(1.2% ASR),证明了在实施上下文中监控的成功,而分解-only监控失败。
英文摘要:
As AI capabilities advance, we increasingly rely on powerful models to decompose complex tasks $\unicode{x2013}$ but what if the decomposer itself is malicious? Factored cognition protocols decompose complex tasks into simpler child tasks: one model creates the decomposition, while other models implement the child tasks in isolation. Prior work uses trusted (weaker but reliable) models for decomposition, which limits usefulness for tasks where decomposition itself is challenging. We introduce Factor($U$,$T$), in which an untrusted (stronger but potentially malicious) model decomposes while trusted models implement child tasks. Can monitors detect malicious activity when observing only natural language task instructions, rather than complete solutions? We baseline and red team Factor($U$,$T$) in control evaluations on BigCodeBench, a dataset of Python coding tasks. Monitors distinguishing malicious from honest decompositions perform poorly (AUROC 0.52) compared to monitors evaluating complete Python solutions (AUROC 0.96). Furthermore, Factor($D$,$U$), which uses a trusted decomposer and monitors concrete child solutions, achieves excellent discrimination (AUROC 0.96) and strong safety (1.2% ASR), demonstrating that implementation-context monitoring succeeds where decomposition-only monitoring fails.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。