发表机构
Institute of Software, Chinese Academy of Sciences (ISCAS); University of Chinese Academy of Sciences; Alibaba Group（中国科学院软件研究所; 中国科学院大学; 阿里巴巴集团）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
HitMem是面向动态环境的分层时序三维记忆框架，通过多模态上下文感知检索机制解决现有三维记忆框架的静态假设缺陷，在Dyna-THOR基准上提升了物体重定位准确率与任务性能。
AI 中文摘要
在动态环境中执行长期任务需要具身智能体维持鲁棒且自适应的三维场景表示。然而，大多数现有的三维记忆框架依赖静态世界假设，当物体因人类活动或未观测到的事件发生位移时，智能体会遭遇记忆-观测冲突，且通常需要代价高昂的几何重计算或低效的全局重探索。为解决该问题，我们提出HitMem，这是一种具有多模态上下文感知检索机制的分层时序三维记忆框架。通过持续感知，HitMem将语义和空间信息统一为捕捉支撑关系的轻量级拓扑图，同时时序衰减机制动态调节记忆活跃度以缓解过时表示的影响。此外，多模态上下文感知检索机制默认使用整合的语义、空间和时序记忆特征过滤候选，当检测到物体位移时则激活专门的两阶段检索流程：该流程结合从外部智能体轨迹推断出的空间约束与基于类别亲和性的语义常识，高效识别高概率候选区域。在我们构建的Dyna-THOR基准上开展的大量评估表明，HitMem可显著提升物体重定位准确率、降低探索成本并增强动态环境中的任务执行性能。
英文摘要
Executing long-term tasks in dynamic environments requires embodied agents to maintain robust and adaptive 3D scene representations. However, most existing 3D memory frameworks rely on static world assumptions. When objects are displaced by human activities or unobserved events, agents encounter memory-observation conflicts and often require costly geometric recomputations or inefficient global re-exploration. To address this, we propose HitMem, a hierarchical temporal 3D memory framework with a multi-modal context-aware retrieval mechanism. Through continuous perception, HitMem unifies semantic and spatial information into a lightweight topological graph that captures support relationships, while a temporal decay mechanism dynamically regulates memory activeness to mitigate the impact of stale representations. In addition, the multi-modal context-aware retrieval mechanism defaults to filtering candidates using integrated semantic, spatial, and temporal memory features, and activates a specialized two-stage retrieval process when object displacement is detected. This process combines spatial constraints inferred from external agent trajectories with semantic common sense grounded in class affinities, efficiently identifying high-probability candidate regions. Extensive evaluations on our constructed Dyna-THOR benchmark demonstrate that HitMem significantly improves object relocation accuracy, reduces exploration costs, and enhances task execution performance in dynamic environments.
CommentsAccepted to ECCV 2026
