arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1910.12980cs.LGstat.ML
收藏

学习可迁移图探索

Learning Transferable Graph Exploration

DeepMind(深度思维);Georgia Institute of Technology(佐治亚理工学院);Google Brain(谷歌大脑);University of Washington(华盛顿大学)

Hanjun Dai, Yujia Li, Chenglong Wang, Rishabh Singh, Po-Sen Huang, Pushmeet Kohli

摘要:

本文考虑了有效探索未知环境的问题,这是人工智能的一个关键挑战。我们提出了一个“学习探索”的框架,我们从环境的分布中学习政策。在测试时,呈现来自相同分布的看不见的环境,该策略旨在将探索策略概括为在有限数量的步骤中访问最大数量的唯一状态。我们特别关注的环境中遇到的许多重要的现实世界中的应用程序,如软件测试和地图建设的图形结构的状态空间。我们制定这个任务作为一个强化学习的问题,“探索”代理的奖励过渡到以前看不见的环境状态,并采用图形结构的内存编码代理的过去轨迹。实验结果表明,我们的方法是非常有效的空间地图的探索,并应用于具有挑战性的问题,覆盖率引导的软件测试领域特定的程序和现实世界的移动应用程序,它优于人工设计的方法,由人类专家。

英文摘要:

This paper considers the problem of efficient exploration of unseen environments, a key challenge in AI. We propose a `learning to explore' framework where we learn a policy from a distribution of environments. At test time, presented with an unseen environment from the same distribution, the policy aims to generalize the exploration strategy to visit the maximum number of unique states in a limited number of steps. We particularly focus on environments with graph-structured state-spaces that are encountered in many important real-world applications like software testing and map building. We formulate this task as a reinforcement learning problem where the `exploration' agent is rewarded for transitioning to previously unseen environment states and employ a graph-structured memory to encode the agent's past trajectory. Experimental results demonstrate that our approach is extremely effective for exploration of spatial maps; and when applied on the challenging problems of coverage-guided software-testing of domain-specific programs and real-world mobile applications, it outperforms methods that have been hand-engineered by human experts.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑