部分可观察辅助博弈中的观察干扰
Observation Interference in Partially Observable Assistance Games
AI总结：
研究部分可观察辅助博弈中AI助手干扰人类观察的动机，证明最优策略、人类即时决策和非理性行为均可能引发干扰，通信渠道可消除部分动机。
摘要：
我们研究部分可观察辅助博弈（POAGs），这是一种人机价值对齐问题的模型，允许人类和AI助手拥有部分观察。出于对AI欺骗的担忧，我们研究了由部分可观察性带来的一种质的新现象：AI助手是否会有动机去干扰人类的观察？首先，我们证明有时最优助手必须采取观察干扰行为，即使人类在最优博弈，且即使存在其他不干扰观察的等效行为。尽管这一结果似乎与单智能体决策中信息价值非负的经典定理相矛盾，但我们通过开发一种定义在整个策略上的干扰概念来解决这一矛盾。这可以看作是信息价值非负的经典结果向合作多智能体环境的扩展。其次，我们证明如果人类仅根据即时结果做决策，助手可能需要通过干扰观察来查询人类的偏好。我们表明，如果人类最优博弈，或引入通信渠道让人类向助手传达偏好，这种干扰动机就会消失。第三，我们表明如果人类按照Boltzmann非理性模型行动，这可能为助手干扰观察创造动机。最后，我们使用实验模型分析AI助手在实践中考虑是否采取观察干扰行为时面临的权衡。
英文摘要：
We study partially observable assistance games (POAGs), a model of the human-AI value alignment problem which allows the human and the AI assistant to have partial observations. Motivated by concerns of AI deception, we study a qualitatively new phenomenon made possible by partial observability: would an AI assistant ever have an incentive to interfere with the human's observations? First, we prove that sometimes an optimal assistant must take observation-interfering actions, even when the human is playing optimally, and even when there are otherwise-equivalent actions available that do not interfere with observations. Though this result seems to contradict the classic theorem from single-agent decision making that the value of information is nonnegative, we resolve this seeming contradiction by developing a notion of interference defined on entire policies. This can be viewed as an extension of the classic result that the value of information is nonnegative into the cooperative multiagent setting. Second, we prove that if the human is simply making decisions based on their immediate outcomes, the assistant might need to interfere with observations as a way to query the human's preferences. We show that this incentive for interference goes away if the human is playing optimally, or if we introduce a communication channel for the human to communicate their preferences to the assistant. Third, we show that if the human acts according to the Boltzmann model of irrationality, this can create an incentive for the assistant to interfere with observations. Finally, we use an experimental model to analyze tradeoffs faced by the AI assistant in practice when considering whether or not to take observation-interfering actions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
