颠覆策略评估:语言模型能否以无状态方式制定策略以颠覆控制协议?
Subversion Strategy Eval: Can language models statelessly strategize to subvert control protocols?
AI总结:
本文提出Subversion Strategy Eval评估套件,通过八个环境和六组支持条件测试语言模型在无状态下制定策略颠覆AI控制协议的能力,发现现有模型均不具备实质性能力,但提供额外支持条件可显著提升性能。
摘要:
AI控制协议是一项旨在防止AI故意造成某些不可接受结果的部署计划。本文研究了AI系统在无状态运行(上下文间无共享内存)时,生成并执行自身策略以颠覆控制协议的能力。为此,AI系统可能需要在每个上下文中可靠地生成最优计划,以校准良好的概率采取行动,并在无通信的情况下与其他自身实例协调计划。我们开发了Subversion Strategy Eval,这是一个包含八个环境、覆盖多种协议和战略能力,以及六组用于隔离单项能力的支持条件的套件。我们在Inspect-AI中实现了该评估并开源发布。我们评估了Claude 3.5模型(包括仅提供帮助的版本)以及OpenAI推理模型。所有模型均未展现出在无状态下制定策略以颠覆控制协议的实质性能力。然而,为模型提供额外支持条件(例如在上下文间共享计划的能力),可显著提升其性能。我们希望这些评估能作为模型何时具备颠覆控制协议能力的领先指标,并放宽AI控制评估中完美战略能力的最坏情况假设。
英文摘要:
An AI control protocol is a plan for usefully deploying AI systems that aims to prevent an AI from intentionally causing some unacceptable outcome. This paper investigates how well AI systems can generate and act on their own strategies for subverting control protocols whilst operating statelessly (without shared memory between contexts). To do this, an AI system may need to reliably generate optimal plans in each context, take actions with well-calibrated probabilities, and coordinate plans with other instances of itself without communicating. We develop Subversion Strategy Eval, a suite of eight environments, covering a range of protocols and strategic capabilities, and six sets of affordances that help isolate individual capabilities. We implement the evaluation in Inspect-AI and release it open-source. We evaluate Claude 3.5 models, including helpful-only versions, as well as OpenAI reasoning models. None of the models demonstrate substantial capability in strategizing to subvert control protocols statelessly. However, providing models with additional affordances, such as the ability to share a plan between contexts, can substantially improve performance. We hope our evaluations can act as a leading indicator for when models are capable of subverting control protocols and also relax the worst-case assumption of perfect strategic ability in AI control evaluations.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。