arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.18639cs.LGmath.OC
收藏

学习具有联合嵌入预测的世界模型以实现规划

JEPA-Bisim: Learning Robust Visual Representations for Planning with Joint-Embedding Predictive World Models

Department of Electrical Engineering, Columbia University, New York, USA.(电气工程系,哥伦比亚大学,纽约,美国)

Leonardo F. Toso, Davit Shadunts, Yunyang Lu, Gloria Geng, Nihal Sharma, Donglin Zhan, Nam H. Nguyen, James Anderson

AI总结:

本文提出了一种联合嵌入预测世界模型,通过引入双模拟编码器增强对慢特征的鲁棒性,提升在减少潜在空间中的导航任务性能。

摘要:

基于高维视觉观察学习的世界模型使智能体能够在潜在空间中直接做出决策和规划,避免了像素级重建。然而,最近的潜在预测架构(JEPAs),包括DINO世界模型(DINO-WM),由于对

英文摘要:

World models learned from high-dimensional visual observations allow agents to make decisions and plan directly in latent space, avoiding pixel-level reconstruction. However, recent latent predictive architectures (JEPAs), including the DINO world model (DINO-WM), display a degradation in test time robustness due to their sensitivity to ``slow features". These include visual variations such as background changes and distractors that are irrelevant to the task being solved. We address this limitation by augmenting the predictive objective with a bisimulation encoder that enforces control-relevant state equivalence, mapping states with similar transition dynamics to nearby latent states while limiting contributions from slow features. We evaluate our model on a navigation task (PointMaze) and on a manipulation task (PushT) under different test-time background changes and visual distractors. Across all benchmarks, our model consistently improves robustness to slow features while operating in a reduced latent space, up to $10\times$ smaller than that of DINO-WM. Moreover, our model is agnostic to the choice of pre-trained visual encoder and maintains robustness when paired with DINOv2, SimDINOv2, and iBOT features.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑