arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.01075cs.LGcs.AIcs.CV
收藏

流等变世界模型:部分观测动态环境的记忆

Flow Equivariant World Models: Structured Memory for Dynamic Environments

Kempner Institute, Harvard University(哈佛大学 Kempner 研究所);ML, Carnegie Mellon University(卡内基梅隆大学 ML 研究所);SEAS, Harvard University(哈佛大学 SEAS 研究所)

Hansen Jin Lillemark, Benhao Huang, Fangneng Zhan, Yilun Du, Thomas Anderson Keller

AI总结:

提出流等变世界建模框架,利用时间参数化对称性在潜在记忆中实现长时程稳定准确的动力学预测,解决部分观测问题。

摘要:

具身系统将世界体验为“流之交响”:多种连续感官输入流与自身运动耦合,并与外部物体的动力学交织。这些感官流和世界的基本动力学遵循平滑的时间参数化对称性,而现有的世界模型忽略了这一点。如果没有尊重这种结构的记忆,部分可观测性对现有方法构成主要障碍:每次观测仅揭示世界的一部分,而未观测区域继续演化。在这项工作中,我们引入了流等变世界建模,这是一个利用潜在记忆中的时间参数化对称性来实现长时程稳定准确动力学预测的框架。潜在记忆随自身运动和推断的外部物体运动等变地移动和变换,使关于视野外区域的信息随时间保持对齐。我们在2D和3D部分观测视频世界建模基准上展示了该框架相对于最先进的扩散、记忆增强和循环世界模型架构的优势。更广泛地说,我们的结果表明,当预测表示按照它们所建模的世界的时间和动力学结构组织时,它们会变得更加强大。项目页面:https://flowequivariantworldmodels.github.io/

英文摘要:

Embodied systems experience the world as 'a symphony of flows': a combination of many continuous streams of sensory input coupled to self-motion, interwoven with the dynamics of external objects. These sensory streams and the underlying dynamics of the world obey smooth, time-parameterized symmetries which existing world models ignore. Without a memory that respects this structure, partial observability presents a major obstacle to existing methods: each observation reveals only a fraction of the world, while unobserved regions continue to evolve. In this work, we introduce Flow Equivariant World Modeling, a framework that leverages time-parameterized symmetries within a latent memory for stable and accurate dynamics prediction over long horizons. The latent memory shifts and transforms equivariantly with self-motion and inferred external object motion, keeping information about out-of-view regions aligned as time progresses. We demonstrate the advantage of this framework over state-of-the-art diffusion, memory-augmented, and recurrent world model architectures on 2D and 3D partially observed video world modeling benchmarks. More broadly, our results suggest that predictive representations become more powerful when they are organized in line with the temporal and dynamical structure of the world they model. Project page: https://flowequivariantworldmodels.github.io/

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑