arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1707.01310cs.AI
收藏

学习设计游戏:强化学习中的战略环境

Learning to Design Games: Strategic Environments in Reinforcement Learning

Peking University(北京大学);University College London(伦敦大学学院);Shanghai Jiao Tong University(上海交通大学)

Haifeng Zhang, Jun Wang, Zhiming Zhou, Weinan Zhang, Ying Wen, Yong Yu, Wenxin Li

摘要:

在典型的强化学习(RL)中,环境被假定为给定的,学习的目标是通过与环境的交互来识别代理采取行动的最优策略。在本文中,我们扩展了这种设置,考虑环境不是给定的,但可控的,可学习的,通过其与代理的互动在同一时间。这种扩展是由现实世界中的环境设计场景,包括游戏设计,购物空间设计和交通信号设计的动机。从理论上讲,我们找到了一个对偶马尔可夫决策过程(MDP)w.r.t.环境对w.r.t.代理,并导出优化参数化环境的策略梯度解决方案。此外,不连续的环境中提出了一个通用的生成框架。我们的迷宫游戏设计任务的实验表明,所提出的算法在生成不同的和具有挑战性的迷宫对各种代理设置的有效性。

英文摘要:

In typical reinforcement learning (RL), the environment is assumed given and the goal of the learning is to identify an optimal policy for the agent taking actions through its interactions with the environment. In this paper, we extend this setting by considering the environment is not given, but controllable and learnable through its interaction with the agent at the same time. This extension is motivated by environment design scenarios in the real-world, including game design, shopping space design and traffic signal design. Theoretically, we find a dual Markov decision process (MDP) w.r.t. the environment to that w.r.t. the agent, and derive a policy gradient solution to optimizing the parametrized environment. Furthermore, discontinuous environments are addressed by a proposed general generative framework. Our experiments on a Maze game design task show the effectiveness of the proposed algorithms in generating diverse and challenging Mazes against various agent settings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑