arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2301.10034cs.AI
收藏

开放世界多任务控制:通过目标感知表示学习与自适应视野预测

Open-World Multi-Task Control Through Goal-Aware Representation Learning and Adaptive Horizon Prediction

Peking University(北京大学);University of California, Los Angeles(加利福尼亚大学洛杉矶分校);Beijing Institute for General Artificial Intelligence(北京通用人工智能研究院)

Shaofei Cai, Zihao Wang, Xiaojian Ma, Anji Liu, Yitao Liang

AI总结:

针对Minecraft开放世界多任务学习中的任务不可区分与非平稳动态两大挑战,提出目标敏感骨干网络与自适应视野预测模块,在20个任务上显著超越基线,部分任务性能翻倍,并展现零样本泛化能力。

摘要:

我们研究在Minecraft中学习目标条件策略的问题,Minecraft是一个流行、广泛可访问但具有挑战性的开放环境,用于开发人类级多任务智能体。我们首先识别出学习此类策略的两个主要挑战:1)由于场景多样性巨大,任务在状态分布上难以区分;2)部分可观测性导致环境动态的非平稳性。为应对第一个挑战,我们提出目标敏感骨干网络(GSB),用于策略中促进目标相关视觉状态表示的出现。为应对第二个挑战,策略进一步由自适应视野预测模块驱动,该模块有助于缓解非平稳动态带来的学习不确定性。在20个Minecraft任务上的实验表明,我们的方法显著优于迄今为止的最佳基线;在许多任务中,我们实现了性能翻倍。我们的消融和探索性研究随后解释了我们的方法如何击败对比方法,并揭示了向新场景(生物群系)零样本泛化的惊人额外优势。我们希望我们的智能体能够为在类似Minecraft的挑战性开放环境中学习目标条件、多任务智能体提供一些启示。

英文摘要:

We study the problem of learning goal-conditioned policies in Minecraft, a popular, widely accessible yet challenging open-ended environment for developing human-level multi-task agents. We first identify two main challenges of learning such policies: 1) the indistinguishability of tasks from the state distribution, due to the vast scene diversity, and 2) the non-stationary nature of environment dynamics caused by partial observability. To tackle the first challenge, we propose Goal-Sensitive Backbone (GSB) for the policy to encourage the emergence of goal-relevant visual state representations. To tackle the second challenge, the policy is further fueled by an adaptive horizon prediction module that helps alleviate the learning uncertainty brought by the non-stationary dynamics. Experiments on 20 Minecraft tasks show that our method significantly outperforms the best baseline so far; in many of them, we double the performance. Our ablation and exploratory studies then explain how our approach beat the counterparts and also unveil the surprising bonus of zero-shot generalization to new scenes (biomes). We hope our agent could help shed some light on learning goal-conditioned, multi-task agents in challenging, open-ended environments like Minecraft.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑