学习保持通勤时间的世界模型用于规划

中文摘要 AI 辅助

世界模型允许智能体在潜在空间中通过选择一系列动作来规划，这些动作最能减少到给定目标状态的距离。因此，规划可以从其距离反映环境中通勤时间的潜在表示中受益。图拉普拉斯算子的谱嵌入空间提供了这样的表示，如果它遵循特定的特征值依赖缩放。不幸的是，在大型连续环境中实例化图拉普拉斯算子是不可行的。自监督学习提供了一种自然的途径来大规模获得这种保持通勤时间的嵌入。然而，在这里我们表明，现有的方法通常鼓励各向同性的表示以防止表示坍缩，往往会破坏“正确的”特征值依赖缩放，导致通勤时间表示不准确。为了解决这个问题，我们引入了保持通勤时间的世界模型（CTWMs），结合了潜在位移预测器和防止坍缩的对数行列式正则化器，在可逆确定性动力学下和预测器的固定点处，可证明地恢复正确缩放的拉普拉斯表示。在数值模拟中，CTWM在多个复杂的连续目标到达基准上匹配或优于任务无关的基线LeWM，同时使用一半的参数。

英文摘要

World models allow agents to plan in latent space by choosing a sequence of actions that most reduces the distance to a given goal state. Thus, planning can benefit from latent representations whose distances mirror commute-times in the environment. The spectral embedding space of the graph Laplacian provides such a representation, if it obeys a specific eigenvalue-dependent scaling. Unfortunately, instantiating the graph Laplacian is intractable in large, continuous environments. Self-supervised learning offers a natural route to such commute-time-preserving embeddings at scale. However, here we show that existing methods, which commonly encourage isotropic representations to prevent representational collapse, tend to degrade the "correct" eigenvalue-dependent scaling, leading to an inaccurate representation of commute times. To address this problem, we introduce Commute-Time-Preserving World Models (CTWMs), combining a latent displacement predictor and a log-determinant regularizer that prevents collapse, which provably recover the correctly scaled Laplacian representation under reversible deterministic dynamics and at the predictor's fixed point. In numerical simulations, CTWM matches or outperforms LeWM, a task-agnostic baseline, on several complex, continuous goal-reaching benchmarks, while using half the parameters.

发表机构

• Friedrich Miescher Institute for Biomedical Research（弗里德里希·米舍尔生物医学研究所）
• University of Basel（巴塞尔大学）

机构由 AI 辅助整理，请以论文原文为准。