arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2611.06475cs.RO
收藏

Robocasa-Mem:用于长 horizon 移动操作的环境与过程记忆基准测试

Robocasa-Mem: Benchmarking Environment and Process Memory in Long-Horizon Mobile Manipulation

University of Science and Technology of China(中国科学技术大学);Tongji University(同济大学);The Chinese University of Hong Kong(香港中文大学);Anhui University(安徽大学);Nanjing University(南京大学)

Sihang Chen, Chen Ye, Wei Chen, Xiaohan Lei, Zhipeng Tang, Sha Zhang, Yuan Zhang, Zixuan Chen, Jiajun Deng, Houqiang Li, Yanyong Zhang

AI总结:

本研究提出含9项任务的RoboCasa-Mem基准,引入行为条件评估,开发HMAP基线,用于评估移动操作中依赖记忆的决策并推动记忆感知策略发展。

摘要:

在现实世界中,移动操作机器人必须执行复杂的物体交互并做出多步骤决策,通常依赖于不再可观测的任务相关信息。然而,现有基准对移动操作中这种依赖记忆的决策评估有限。为解决这一差距,我们引入 RoboCasa-Mem,这一新基准包含9项移动操作任务,要求智能体在关键决策点利用交互历史。具体而言,这些任务分为两类记忆:一类是环境记忆,用于捕捉空间位置、物体属性和物体关系;另一类是过程记忆,用于捕捉单任务进度和多任务调度。特别地,我们引入行为条件评估:对于具有多个有效执行路径的任务,评估器不固定单一目标或轨迹,而是从机器人的早期动作中推导后续任务约束,这使我们能够评估后续行为是否与机器人自身的交互历史一致,同时减少基于固定目标分配的捷径。我们提供1800个成功演示用于策略训练,还在该基准上开发了基线模型,即分层记忆增强策略(Hierarchical Memory-Augmented Policy,HMAP):其高层规划器使用结构化实体和过程记忆生成子任务,低层VLA控制器使用子目标条件查询将VGGT-Omega几何信息融入动作生成。RoboCasa-Mem与HMAP基线共同支持对依赖记忆行为的系统评估,以及记忆感知移动操作策略的开发。

英文摘要:

In the real world, mobile manipulation robots must perform complex object interactions and make multi-step decisions, often relying on task-relevant information that is no longer observable. However, existing benchmarks provide limited evaluation of such memory-dependent decision-making in mobile manipulation. To address this gap, we introduce RoboCasa-Mem, a new benchmark comprising nine mobile manipulation tasks that require agents to use interaction history at key decision points. Specifically, the tasks are grouped into two categories of memory: One is environment memory, which captures spatial locations, object attributes, and object relationships. The other is process memory, which captures single-task progress and multi-task scheduling. Particularly, we introduce Behavior-Conditioned Evaluation. For tasks with multiple valid execution paths, instead of fixing a single target or trajectory, the evaluator derives later task constraints from the robot's earlier actions. This allows us to evaluate whether later behavior is consistent with the robot's own interaction history while reducing shortcuts based on fixed target assignments. We provide 1,800 successful demonstrations for policy training. We also develop a baseline on our benchmark, namely Hierarchical Memory-Augmented Policy(HMAP). Its high-level planner uses structured entity and process memories to generate subtasks. Its low-level VLA controller uses subgoal-conditioned queries to incorporate VGGT-Omega geometry into action generation. Together, RoboCasa-Mem and HMAP baseline support systematic evaluation of memory-dependent behavior and the development of memory-aware mobile manipulation policies.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑