通过软状态不变世界模型简化潜在动力学
Simplifying Latent Dynamics with Softly State-Invariant World Models
AI总结：
针对现有世界模型无法保证动作效果系统性表征的问题，提出软状态不变的简约潜在空间模型PLSM，通过最小化潜在状态与动作引发的潜在变化间的互信息正则化动力学，在预测、规划及无模型强化学习任务中提升了准确性、泛化性和性能。
摘要：
为通过基于模型的推理或规划解决控制问题，智能体需要了解其动作如何影响世界状态。智能体可支配的动作通常会以系统性的方式改变环境状态。然而，现有的世界建模技术无法保证动作的效果以这种系统性方式被表征。我们提出了Parsimonious Latent Space Model（PLSM，简约潜在空间模型），这是一种对潜在动力学施加正则化的世界模型，旨在让智能体动作的效果更具可预测性。我们的方法最小化潜在状态与动作在智能体潜在状态中引发的变化之间的互信息，进而最小化状态对动力学的依赖，使世界模型具备软状态不变性。我们将PLSM与三类不同的模型结合，分别用于：i）未来潜在状态预测，ii）规划，以及iii）无模型强化学习。研究发现，我们的正则化方法提升了下游任务的准确性、泛化能力和性能，凸显了在世界模型中系统性处理动作的重要性。
英文摘要：
To solve control problems via model-based reasoning or planning, an agent needs to know how its actions affect the state of the world. The actions an agent has at its disposal often change the state of the environment in systematic ways. However, existing techniques for world modelling do not guarantee that the effect of actions are represented in such systematic ways. We introduce the Parsimonious Latent Space Model (PLSM), a world model that regularizes the latent dynamics to make the effect of the agent's actions more predictable. Our approach minimizes the mutual information between latent states and the change that an action produces in the agent's latent state, in turn minimizing the dependence the state has on the dynamics. This makes the world model softly state-invariant. We combine PLSM with different model classes used for i) future latent state prediction, ii) planning, and iii) model-free reinforcement learning. We find that our regularization improves accuracy, generalization, and performance in downstream tasks, highlighting the importance of systematic treatment of actions in world models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
