arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2303.07437cs.LGcs.AI
收藏

部分可观测Atari博弈中的无监督表示学习

Unsupervised Representation Learning in Partially Observable Atari Games

University of Oslo(奥斯陆大学);University of Agder(阿格德大学);Oslo Metropolitan University(奥斯陆城市大学)

Li Meng, Morten Goodwin, Anis Yazidi, Paal Engelstad

摘要:

状态表示学习旨在捕捉环境中潜在的因素。在以前的状态表示学习研究中,对比方法比生成模型表现得更好。尽管一些研究者已经意识到了掩码图像建模和对比表示学习之间的联系,但他们的努力主要集中在使用掩模作为增强技术来更好地表示潜在的生成因素。强化学习中的部分可观察环境尚未使用无监督状态表示学习方法进行仔细研究。在这篇文章中,我们创建了一个无监督的状态表示学习计划的部分可观察的状态。我们在以前的Atari 2600框架上进行了实验,该框架旨在评估表示学习模型。一种名为Spatiotemporal DeepInfomax(ST-DIM)的对比方法在这个基准测试中表现出了最先进的性能,但仍然不如其监督的对应方法。我们的方法在环境不完全可观察时改进了ST-DIM,并获得了比监督学习更高的F1分数和准确性分数。我们的方法在类别上的平均准确率得分为~ 66%,而监督学习为~38%。平均F1评分为~64%至~ 33%。

英文摘要:

State representation learning aims to capture latent factors of an environment. Contrastive methods have performed better than generative models in previous state representation learning research. Although some researchers realize the connections between masked image modeling and contrastive representation learning, the effort is focused on using masks as an augmentation technique to represent the latent generative factors better. Partially observable environments in reinforcement learning have not yet been carefully studied using unsupervised state representation learning methods. In this article, we create an unsupervised state representation learning scheme for partially observable states. We conducted our experiment on a previous Atari 2600 framework designed to evaluate representation learning models. A contrastive method called Spatiotemporal DeepInfomax (ST-DIM) has shown state-of-the-art performance on this benchmark but remains inferior to its supervised counterpart. Our approach improves ST-DIM when the environment is not fully observable and achieves higher F1 scores and accuracy scores than the supervised learning counterpart. The mean accuracy score averaged over categories of our approach is ~66%, compared to ~38% of supervised learning. The mean F1 score is ~64% to ~33%.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑