arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1808.07645cs.HCcs.AIcs.CL
收藏

基于策略的强化学习玩20个问题游戏

Playing 20 Question Game with Policy-Based Reinforcement Learning

Peking University(北京大学);Microsoft Corporation(微软公司);Microsoft Development Co., Ltd(微软开发有限公司)

Huang Hu, Xianchao Wu, Bingfeng Luo, Chongyang Tao, Can Xu, Wei Wu, Zhan Chen

AI总结:

提出基于策略的强化学习方法,使提问者通过与用户持续交互学习最优问题选择策略,并利用奖励网络估计信息奖励,无需知识库且对噪声答案鲁棒。

摘要:

20个问题(Q20)游戏是一种著名的游戏,鼓励演绎推理和创造力。在游戏中,回答者首先想一个对象,如一个著名人物或一种动物。然后提问者通过问20个问题来猜测对象。在Q20游戏系统中,用户被视为回答者,而系统本身充当提问者,需要良好的问题选择策略来找出正确的对象并赢得游戏。然而,由于游戏环境的复杂性和波动性,最优的问题选择策略很难推导。在本文中,我们提出了一种新颖的基于策略的强化学习(RL)方法,使提问者代理能够通过与用户的持续交互学习最优的问题选择策略。为了促进训练,我们还提出使用奖励网络来估计更具信息量的奖励。与以前的方法相比,我们的RL方法对噪声答案具有鲁棒性,并且不依赖于对象的知识库。实验结果表明,我们的RL方法明显优于基于熵的工程系统,并且在无噪声模拟环境中具有竞争力的性能。

英文摘要:

The 20 Questions (Q20) game is a well known game which encourages deductive reasoning and creativity. In the game, the answerer first thinks of an object such as a famous person or a kind of animal. Then the questioner tries to guess the object by asking 20 questions. In a Q20 game system, the user is considered as the answerer while the system itself acts as the questioner which requires a good strategy of question selection to figure out the correct object and win the game. However, the optimal policy of question selection is hard to be derived due to the complexity and volatility of the game environment. In this paper, we propose a novel policy-based Reinforcement Learning (RL) method, which enables the questioner agent to learn the optimal policy of question selection through continuous interactions with users. To facilitate training, we also propose to use a reward network to estimate the more informative reward. Compared to previous methods, our RL method is robust to noisy answers and does not rely on the Knowledge Base of objects. Experimental results show that our RL method clearly outperforms an entropy-based engineering system and has competitive performance in a noisy-free simulation environment.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑