arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2203.00494cs.LGcs.AIcs.SYeess.SY
收藏

DreamingV2:基于无重构离散世界模型的强化学习

DreamingV2: Reinforcement Learning with Discrete World Models without Reconstruction

Panasonic Corporation(松下公司);Ritsumeikan University(立命馆大学)

Masashi Okada, Tadahiro Taniguchi

摘要:

本文提出了一种新的强化学习方法与世界模型,DreamingV2,DreamerV2和Dreaming的合作扩展。DreamerV2是一种基于模型的像素强化学习,它使用离散的世界模型来表示具有分类变量的潜在状态。做梦也是像素的强化学习的一种形式,它试图通过涉及无重构的对比学习目标来避免一般世界模型训练中的自动编码过程。所提出的DreamingV2是一种新颖的方法,采用DreamingV2的离散表示和Dreaming的无重构目标。与DreamerV2和其他最近的基于模型的方法相比,DreamingV2在五个模拟的具有挑战性的3D机器人手臂任务上取得了最好的成绩。我们相信DreamingV2将是机器人学习的一个可靠的解决方案,因为它的离散表示适合描述不连续的环境,并且无重构的方式很好地管理复杂的视觉观察。

英文摘要:

The present paper proposes a novel reinforcement learning method with world models, DreamingV2, a collaborative extension of DreamerV2 and Dreaming. DreamerV2 is a cutting-edge model-based reinforcement learning from pixels that uses discrete world models to represent latent states with categorical variables. Dreaming is also a form of reinforcement learning from pixels that attempts to avoid the autoencoding process in general world model training by involving a reconstruction-free contrastive learning objective. The proposed DreamingV2 is a novel approach of adopting both the discrete representation of DreamingV2 and the reconstruction-free objective of Dreaming. Compared to DreamerV2 and other recent model-based methods without reconstruction, DreamingV2 achieves the best scores on five simulated challenging 3D robot arm tasks. We believe that DreamingV2 will be a reliable solution for robot learning since its discrete representation is suitable to describe discontinuous environments, and the reconstruction-free fashion well manages complex vision observations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑