arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2512.02157cs.CRcs.CL
收藏

因子(T,U):因子认知强化对不可信AI的监控

Factor(T,U): Factored Cognition Strengthens Monitoring of Untrusted AI

Independent(独立研究者)

Aaron Sandoval, Cody Rushing

AI总结:

本文提出基于因子认知的AI控制协议,通过分解任务和独立解决子任务来提高安全性,实验显示其在提升监控性能和降低后门风险方面具有显著效果。

摘要:

AI控制领域旨在开发稳健的控制协议,为可能有意破坏的不可信AI提供部署保障。然而,现有依赖较弱监控器检测不安全行为的协议往往在复杂任务上失效,因为这些任务超出监控器的理解范围。我们开发了基于因子认知的控制协议,其中可信模型分解任务,不可信模型独立解决每个子任务,然后将结果重新组装成完整解决方案。这些协议可通过多种方式提高安全性,例如通过简化监控器的上下文,或通过掩盖环境中的漏洞。我们在APPS编码设置中实施了我们的协议,并通过对抗性GPT-4.1 Nano的红队测试来测试其效果。我们发现:(i) 在可信监控协议中添加因子认知可将安全性从41%提升到63%;(ii) 安全性提升是因为监控性能的提升;(iii) 因子认知使有能力的LLMs在APPS中编写后门没有更难。尽管我们的协议在APPS中表现出较低的实用性,但它们在更复杂的任务中具有潜力。

英文摘要:

The field of AI Control seeks to develop robust control protocols, deployment safeguards for untrusted AI which may be intentionally subversive. However, existing protocols that rely on weaker monitors to detect unsafe behavior often fail on complex tasks beyond the monitor's comprehension. We develop control protocols based on factored cognition, in which a trusted model decomposes a task, an untrusted model solves each resultant child task in isolation, and the results are reassembled into a full solution. These protocols may improve safety by several means, such as by simplifying the context for monitors, or by obscuring vulnerabilities in the environment. We implement our protocols in the APPS coding setting and red team them against backdoor attempts from an adversarial GPT-4.1 Nano. We find that: (i) Adding factored cognition to a trusted monitoring protocol can boost safety from 41% to 63%; (ii) Safety improves because monitor performance improves; (iii) Factored cognition makes it no harder for capable LLMs to write backdoors in APPS. While our protocols show low usefulness in APPS, they hold promise for more complex tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑