HiMem-WAM: 用于机器人操作的分层记忆门控世界动作模型
HiMem-WAM: Hierarchical Memory-Gated World Action Models for Robotic Manipulation
- The University of Hong Kong（香港大学）
- INFIFORCE
- Huazhong University of Science and Technology（华中科技大学）
- Tsinghua University（清华大学）
- Wuhan University（武汉大学）
- Southern University of Science and Technology（南方科技大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
提出分层记忆门控世界动作模型HiMem-WAM，通过分层潜在动作框架和边界触发记忆更新，提升长时域机器人操作的任务相关记忆与泛化鲁棒性。
AI中文摘要：
世界动作模型（WAM）已成为具身智能的一种新的强大范式，学习与动作相关的视觉动态，显著增强了泛化性和鲁棒性。然而，现有的WAM在长时域机器人操作中仍难以处理任务相关记忆。为了解决这个问题，我们提出了HiMem-WAM，一种分层记忆门控WAM，它集成了以运动为中心的潜在动作、高级技能潜在变量和边界触发的记忆更新。具体来说，我们开发了一个分层潜在动作框架，共同学习低级运动和高级技能潜在变量，提供结构化的时间抽象。同时，边界感知记忆门在预测的技能转换处写入紧凑的任务状态，无需在测试时生成未来视频或光流估计即可实现因果推理。在LIBERO、LIBERO-PLUS、RMBench和真实世界任务上的评估表明，HiMem-WAM的分层潜在变量提高了部署扰动下的鲁棒性，而记忆模块显著有益于依赖记忆的长时域操作。
英文摘要：
World Action Models (WAMs) have emerged as a new powerful paradigm for embodied intelligence, learning action-relevant visual dynamics that significantly enhance generalization and robustness. However, existing WAMs still struggle with task-relevant memory in long-horizon robotic manipulation. To address this, we present HiMem-WAM, a Hierarchical Memory-Gated WAM that integrates motion-centric latent actions, high-level skill latents, and boundary-triggered memory updates. Specifically, we develop a hierarchical latent action framework that jointly learns low-level motion and high-level skill latents, providing structured temporal abstraction. Meanwhile, a boundary-aware memory gate writes compact task states at predicted skill transitions, enabling causal inference without test-time generation of future video or optical flow estimation. Evaluated on LIBERO, LIBERO-PLUS, RMBench and real-world tasks, HiMem-WAM shows that hierarchical latents improve robustness under deployment perturbations, and the memory module substantially benefits memory-dependent long-horizon manipulation.
