arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2101.11071cs.LGcs.AIstat.ML
收藏

MineRL 2020年使用人类先验知识的样本有效强化学习竞赛

The MineRL 2020 Competition on Sample Efficient Reinforcement Learning using Human Priors

Carnegie Mellon University(卡内基梅隆大学);OpenAI Inc.(OpenAI公司);Preferred Networks, Inc.(Preferred Networks公司);Microsoft Research(微软研究院);AIcrowd SA(AIcrowd公司);DeepMind(DeepMind公司)
William H. Guss, Mario Ynocente Castro, Sam Devlin, Brandon Houghton, Noboru Sean Kuno, Crissman Loomis, Stephanie Milani, Sharada Mohanty, Keisuke Nakata, Rusl… 展开作者

William H. Guss, Mario Ynocente Castro, Sam Devlin, Brandon Houghton, Noboru Sean Kuno, Crissman Loomis, Stephanie Milani, Sharada Mohanty, Keisuke Nakata, Ruslan Salakhutdinov, John Schulman, Shinya Shiroshita, Nicholay Topin, Avinash Ummadisingu, Oriol Vinyals

摘要:

尽管深度强化学习在许多困难领域取得了突破,但这些成功需要不断增加的样本数量,只有越来越少的AI社区能够获得它们的开发。解决这些限制需要新的,样品效率高的方法。为了促进这一方向的研究,我们提出了MineRL竞赛的第二次迭代。比赛的主要目标是促进算法的开发,这些算法可以有效地利用人类演示,从而大大减少解决复杂,分层和稀疏环境所需的样本数量。为此,参与者在有限的环境样本复杂度预算下竞争开发解决Minecraft中MineRL ObtainDiamond任务的系统,这是一个需要长期规划,分层控制和有效探索方法的顺序决策环境。比赛分为两轮,为参赛者提供了几个配对版本的数据集和环境,以及不同的游戏纹理和着色器。在每一轮结束时,竞争对手将其学习算法的容器化版本提交到AICrowd平台,在那里他们在预先指定的硬件平台上从零开始,在一个保持不变的网络环境对上进行总共4天的训练。在NeurIPS 2019 MineRL竞赛的后续迭代中,我们实施了新功能,以扩大竞赛的规模和范围。为了回应之前参与者的反馈,我们引入了第二个次要轨道,重点关注除测试期间外无法访问任何类型的环境交互的解决方案。此外,我们的目标是通过实施几个新的竞争机制,包括行动空间随机化和desemantization的观察和行动,以促进域不可知的意见书。

英文摘要:

Although deep reinforcement learning has led to breakthroughs in many difficult domains, these successes have required an ever-increasing number of samples, affording only a shrinking segment of the AI community access to their development. Resolution of these limitations requires new, sample-efficient methods. To facilitate research in this direction, we propose this second iteration of the MineRL Competition. The primary goal of the competition is to foster the development of algorithms which can efficiently leverage human demonstrations to drastically reduce the number of samples needed to solve complex, hierarchical, and sparse environments. To that end, participants compete under a limited environment sample-complexity budget to develop systems which solve the MineRL ObtainDiamond task in Minecraft, a sequential decision making environment requiring long-term planning, hierarchical control, and efficient exploration methods. The competition is structured into two rounds in which competitors are provided several paired versions of the dataset and environment with different game textures and shaders. At the end of each round, competitors submit containerized versions of their learning algorithms to the AIcrowd platform where they are trained from scratch on a hold-out dataset-environment pair for a total of 4-days on a pre-specified hardware platform. In this follow-up iteration to the NeurIPS 2019 MineRL Competition, we implement new features to expand the scale and reach of the competition. In response to the feedback of the previous participants, we introduce a second minor track focusing on solutions without access to environment interactions of any kind except during test-time. Further we aim to prompt domain agnostic submissions by implementing several novel competition mechanics including action-space randomization and desemantization of observations and actions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑