arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.06445cs.CVcs.AI
收藏

通过训练智能体探索学习生成式交互环境

Learning Generative Interactive Environments By Trained Agent Exploration

INSAIT;Sofia University "St. Kliment Ohridski"(索菲亚大学"圣·克利门特·奥赫里德斯基")

Naser Kazemi, Nedko Savov, Danda Paudel, Luc Van Gool

AI总结:

本文提出用强化学习智能体探索替代随机智能体来生成训练数据,并发布GenieRedux及GenieRedux-G模型,在Coinrun等环境中验证了其在视觉保真度和可控性上的提升。

摘要:

世界模型在解释和模拟复杂环境的规则与动作方面日益关键。最近的模型Genie擅长从视觉多样的环境中学习,但依赖昂贵的人工采集数据。我们观察到,其使用随机智能体的替代方法在探索环境方面过于局限。我们提出通过采用基于强化学习的智能体进行数据生成来改进该模型。这种方法能产生多样化的数据集,从而增强模型在各种场景和环境中真实动作下的适应能力与表现。在本文中,我们首先发布模型GenieRedux——一个基于Genie的实现。此外,我们引入GenieRedux-G,这是一个变体,利用智能体现成的动作在验证过程中分解动作预测的不确定性。我们的评估(包括对Coinrun案例研究的复现)表明,GenieRedux-G通过使用训练智能体探索,实现了更优的视觉保真度和可控性。所提出的方法具有可复现性、可扩展性,并能适应新型环境。我们的代码库可在https://github.com/insait-institute/GenieRedux获取。

英文摘要:

World models are increasingly pivotal in interpreting and simulating the rules and actions of complex environments. Genie, a recent model, excels at learning from visually diverse environments but relies on costly human-collected data. We observe that their alternative method of using random agents is too limited to explore the environment. We propose to improve the model by employing reinforcement learning based agents for data generation. This approach produces diverse datasets that enhance the model's ability to adapt and perform well across various scenarios and realistic actions within the environment. In this paper, we first release the model GenieRedux - an implementation based on Genie. Additionally, we introduce GenieRedux-G, a variant that uses the agent's readily available actions to factor out action prediction uncertainty during validation. Our evaluation, including a replication of the Coinrun case study, shows that GenieRedux-G achieves superior visual fidelity and controllability using the trained agent exploration. The proposed approach is reproducable, scalable and adaptable to new types of environments. Our codebase is available at https://github.com/insait-institute/GenieRedux .

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑