MoWM: 通过潜在到像素特征调制的混合世界模型实现具身规划
MoWM: Mixture-of-World-Models for Embodied Planning via Latent-to-Pixel Feature Modulation
AI总结：
MoWM通过融合潜在世界模型与像素特征，提升具身规划中动作解码的精度与泛化能力。
摘要：
具身动作规划是机器人领域的核心挑战，要求模型能够从视觉观测和语言指令中生成精确的动作。尽管视频生成世界模型颇具前景，但其对像素级重建的依赖往往引入视觉冗余，阻碍了动作解码和泛化。潜在世界模型提供了一种紧凑、运动感知的表示，但忽略了对精确操作至关重要的细粒度细节。为克服这些限制，我们提出了MoWM，一种混合世界模型框架，通过融合混合世界模型的表示来实现具身动作规划。我们的方法结合了运动感知的潜在世界模型特征与像素空间特征，使MoWM能够强调与动作相关的视觉细节以进行动作解码。在CALVIN和现实世界操作任务上的广泛评估表明，我们的方法在任务成功率和泛化能力方面均达到最先进的水平。我们还对每个特征空间的优势进行了全面分析，为未来具身规划研究提供了有价值的见解。代码可在：https://github.com/tsinghua-fib-lab/MoWM获取。
英文摘要：
Embodied action planning is a core challenge in robotics, requiring models to generate precise actions from visual observations and language instructions. While video generation world models are promising, their reliance on pixel-level reconstruction often introduces visual redundancies that hinder action decoding and generalization. Latent world models offer a compact, motion-aware representation, but overlook the fine-grained details critical for precise manipulation. To overcome these limitations, we propose MoWM, a mixture-of-world-model framework that fuses representations from hybrid world models for embodied action planning. Our approach combines motion-aware latent world model features with pixel-space features, enabling MoWM to emphasize action-relevant visual details for action decoding. Extensive evaluations on the CALVIN and real-world manipulation tasks demonstrate that our method achieves state-of-the-art task success rates and superior generalization. We also provide a comprehensive analysis of the strengths of each feature space, offering valuable insights for future research in embodied planning. The code is available at: https://github.com/tsinghua-fib-lab/MoWM.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
