arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2204.11142cs.LGcs.AI
收藏

基于图神经网络的Agent在Google Research Football中的应用

Graph Neural Network based Agent in Google Research Football

Central South University of Forestry and Technology(中南林业科技大学);Fordham University(福特汉姆大学);Zhejiang University(浙江大学)

Yizhan Niu, Jinglong Liu, Yuhao Shi, Jiren Zhu

摘要:

深度神经网络(DNN)可以近似价值函数或策略用于强化学习,这使得强化学习算法更强大。然而,在强化学习的特定情况下,一些DNN,如卷积神经网络(CNN),无法提取足够的信息或需要太长时间才能从输入中获得足够的特征。例如,Google Research Football的输入数据是球员位置的小地图,这是一个训练代理踢足球的强化学习环境。这些信息不仅包含在玩家的坐标中,还包含在不同玩家之间的关系中。CNN既不能提取足够的信息,也不需要太长的训练时间。为了解决这个问题,本文提出了一种以图神经网络(GNN)为模型的深度q学习网络(DQN)。GNN将输入数据转换为更好地表示足球运动员位置的图,以便提取不同运动员之间交互的更多信息。通过两个GNN来近似其本地和目标价值函数,该DQN允许玩家通过使用价值函数来了解每个预期动作的预期价值,从而从他们的经验中学习。所提出的模型证明了GNN在足球比赛中的强大功能,其表现优于其他DRL模型,步骤明显减少。

英文摘要:

Deep neural networks (DNN) can approximate value functions or policies for reinforcement learning, which makes the reinforcement learning algorithms more powerful. However, some DNNs, such as convolutional neural networks (CNN), cannot extract enough information or take too long to obtain enough features from the inputs under specific circumstances of reinforcement learning. For example, the input data of Google Research Football, a reinforcement learning environment which trains agents to play football, is the small map of players' locations. The information is contained not only in the coordinates of players, but also in the relationships between different players. CNNs can neither extract enough information nor take too long to train. To address this issue, this paper proposes a deep q-learning network (DQN) with a graph neural network (GNN) as its model. The GNN transforms the input data into a graph which better represents the football players' locations so that it extracts more information of the interactions between different players. With two GNNs to approximate its local and target value functions, this DQN allows players to learn from their experience by using value functions to see the prospective value of each intended action. The proposed model demonstrated the power of GNN in the football game by outperforming other DRL models with significantly fewer steps.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑