在复杂环境中避免副作用
Avoiding Side Effects in Complex Environments
摘要:
奖励函数规范可能很困难。奖励制作小部件的代理可能很容易,但惩罚大量可能的负面影响很难。在玩具环境中,属性效用保持(AUP)通过惩罚实现随机生成目标的能力的变化来避免副作用。我们将这种方法扩展到大型的,随机生成的环境的基础上康威的生命游戏。通过为单个随机生成的奖励函数保留最优值,AUP在导致代理完成指定任务并避免许多副作用的同时产生适度的开销。视频和代码可在https://avoiding-side-effects.github.io/上获得。
英文摘要:
Reward function specification can be difficult. Rewarding the agent for making a widget may be easy, but penalizing the multitude of possible negative side effects is hard. In toy environments, Attainable Utility Preservation (AUP) avoided side effects by penalizing shifts in the ability to achieve randomly generated goals. We scale this approach to large, randomly generated environments based on Conway's Game of Life. By preserving optimal value for a single randomly generated reward function, AUP incurs modest overhead while leading the agent to complete the specified task and avoid many side effects. Videos and code are available at https://avoiding-side-effects.github.io/.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。