梦见众多世界：学习上下文世界模型有助于零样本泛化
Dreaming of Many Worlds: Learning Contextual World Models Aids Zero-Shot Generalization
AI总结：
本文提出 cRSSM，将动力学上下文融入 Dreamer v3 世界模型，以在 cRL 设定下提升策略对未见动力学的零样本泛化，并在 CARL 上验证其效果。
摘要：
对未见动力学进行零样本泛化（ZSG）是打造通用具身智能体的一项重大挑战。为应对这一更广泛的挑战，我们从较简单的情境强化学习（cRL）设定入手，假设可以观测到用于参数化系统动力学变化的上下文值，例如机器人的质量或尺寸，而不对马尔可夫状态的可观测性作出进一步简化假设。为实现对上下文未见变化的 ZSG 目标，我们提出了情境循环状态空间模型（cRSSM），该模型对 Dreamer (v3)（Hafner 等人，2023）的世界模型进行了修改。这使世界模型能够整合上下文，从而从观测中推断潜在马尔可夫状态并对潜在动力学建模。我们的方法在专为研究情境 RL 而设计的 CARL 基准套件中的两个任务上进行了评估。实验表明，这种对上下文的系统性整合提升了在世界模型“梦境”上训练的策略的 ZSG 能力。我们进一步从定性上发现，该方法使 Dreamer 能够将潜在状态与上下文分离开来，从而将其梦境外推到未见上下文的众多世界中。所有实验的代码可在 https://github.com/sai-prasanna/dreaming_of_many_worlds 获取。
英文摘要：
Zero-shot generalization (ZSG) to unseen dynamics is a major challenge for creating generally capable embodied agents. To address the broader challenge, we start with the simpler setting of contextual reinforcement learning (cRL), assuming observability of the context values that parameterize the variation in the system's dynamics, such as the mass or dimensions of a robot, without making further simplifying assumptions about the observability of the Markovian state. Toward the goal of ZSG to unseen variation in context, we propose the contextual recurrent state-space model (cRSSM), which introduces changes to the world model of Dreamer (v3) (Hafner et al., 2023). This allows the world model to incorporate context for inferring latent Markovian states from the observations and modeling the latent dynamics. Our approach is evaluated on two tasks from the CARL benchmark suite, which is tailored to study contextual RL. Our experiments show that such systematic incorporation of the context improves the ZSG of the policies trained on the "dreams" of the world model. We further find qualitatively that our approach allows Dreamer to disentangle the latent state from context, allowing it to extrapolate its dreams to the many worlds of unseen contexts. The code for all our experiments is available at https://github.com/sai-prasanna/dreaming_of_many_worlds.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
