arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2302.02119cs.AIcs.LG
收藏

通过自我博弈实现多样性诱导的环境设计

Diversity Induced Environment Design via Self-Play

Singapore Management University(新加坡管理大学)

Dexun Li, Wenjun Li, Pradeep Varakantham

AI总结:

本文提出DivSP方法,在无监督环境设计中引入多样性度量,并结合自我博弈技术提升采样效率,从而生成更有效的训练环境,性能优于现有方法。

摘要:

近期关于设计合适环境分布的研究在训练高效通用智能体方面展现出潜力。其成功部分归因于一种自适应课程学习形式,该形式在智能体能力边界处生成环境实例(或关卡)。然而,此类环境设计框架在具有挑战性的设计空间中往往难以找到有效关卡,并且需要与环境进行昂贵的交互。在本文中,我们旨在将多样性引入无监督环境设计(UED)框架。具体而言,我们提出一种任务无关的方法,用于识别给定关卡中具有代表性的观测/隐藏状态。该方法的输出随后被用于表征两个关卡之间的多样性,我们证明这对有效性能至关重要。此外,为提高采样效率,我们引入自我博弈技术,使环境生成器能够自动生成对训练智能体大有裨益的环境。定量上,我们的方法——通过自我博弈实现多样性诱导的环境设计(DivSP)——展现出优于现有方法的令人信服的性能。

英文摘要:

Recent work on designing an appropriate distribution of environments has shown promise for training effective generally capable agents. Its success is partly because of a form of adaptive curriculum learning that generates environment instances (or levels) at the frontier of the agent's capabilities. However, such an environment design framework often struggles to find effective levels in challenging design spaces and requires costly interactions with the environment. In this paper, we aim to introduce diversity in the Unsupervised Environment Design (UED) framework. Specifically, we propose a task-agnostic method to identify observed/hidden states that are representative of a given level. The outcome of this method is then utilized to characterize the diversity between two levels, which as we show can be crucial to effective performance. In addition, to improve sampling efficiency, we incorporate the self-play technique that allows the environment generator to automatically generate environments that are of great benefit to the training agent. Quantitatively, our approach, Diversity-induced Environment Design via Self-Play (DivSP), shows compelling performance over existing methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑