世界即图：通过潜在空间图进行关系世界建模
World-as-Graph: Relational World Modeling Through Latent Space Graphs
浏览论文内容
中文总结 AI 辅助
提出世界即图（WAG），一种基于图的对象中心世界模型，通过关系感知结构归纳和对象中心记忆转换，在JEPA框架中引入关系归纳偏置，实现显式关系建模与自回归预测，在视觉推理和机器人操作任务中表现优越。
中文摘要 AI 辅助
世界模型旨在学习真实世界环境的表示并预测其未来演化。最近的以对象为中心的世界模型通过将视觉场景表示为对象级潜在状态的集合取得了显著进展，但对象间的关系往往仅被隐式捕获，这限制了显式关系和时序结构建模以及以对象为中心的动态记忆建模。为解决这些挑战，我们提出了世界即图（WAG），一种基于图的对象中心世界模型，将关系归纳偏置引入JEPA风格的预测表示学习中。所提出的WAG包含两个主要模块：（1）关系感知结构归纳，从对象中心槽构建时变潜在图，并设计关系感知对象掩蔽策略以引导潜在空间中关系对象表示学习；（2）对象中心记忆转换，通过结合邻近对象的关系信息与历史记忆来维护和更新对象级动态状态，实现有效的自回归未来预测。在视觉推理和机器人操作任务上的大量实验证明了所提出的WAG的优越性能。
英文摘要
World models aim to learn representations of real-world environments and predict their future evolution. Recent object-centric world models have made expressive progress by representing visual scenes as sets of object-level latent states, but object-object relations are often captured only implicitly, which limits explicit relational and temporal structure modeling and object-centric dynamic memory modeling. To address such challenges, we propose World-As-Graph (WAG), a graph-based object-centric world model that introduces relational inductive bias into JEPA-style predictive representation learning. The proposed WAG contains two main modules: (1) Relation-aware structure induction, which constructs time-varying latent graphs from object-centric slots and designs relation-aware object masking policies to guide relational object representation learning in latent space; (2) Object-centric memory transition, which maintains and updates object-level dynamic states by combining relational information from neighboring objects with historical memory, enabling effective autoregressive future prediction. Extensive experiments on both visual reasoning and robotic manipulation tasks could demonstrate the superior performance of our proposed WAG.
发表机构
- RMIT University（皇家墨尔本理工大学）
- Monash University（莫纳什大学）
- KStelrix（KStelrix公司）
- The University of Melbourne（墨尔本大学）
机构由 AI 辅助整理，请以论文原文为准。
