看见更大的图景：用于移动操作策略学习的3D潜在映射
Seeing the Bigger Picture: 3D Latent Mapping for Mobile Manipulation Policy Learning
AI总结：
本文提出SBP方法，通过3D潜在映射提升移动操作策略的学习性能，实现更强的空间和时间推理能力。
摘要：
在本文中，我们证明了使用3D潜在地图的移动操作策略在空间和时间推理方面比仅依赖图像的策略更强。我们引入了Seeing the Bigger Picture（SBP），一种端到端的策略学习方法，直接在3D的潜在特征地图上运行。在SBP中，地图超越了机器人当前的视野范围，并在长时间范围内汇总观察。我们的映射方法逐步将多视角观察融合到场景特定的潜在特征网格中。一个预训练的、场景无关的解码器从这些特征中重建目标嵌入，并在任务执行过程中启用对地图特征的在线优化。一个可训练的策略，通过行为克隆或强化学习进行训练，将潜在地图视为状态变量，并利用3D特征聚合器从地图中获取全局上下文。我们在场景级移动操作和连续桌面操作任务上评估了SBP。我们的实验表明，SBP（i）在场景层面进行全局推理，（ii）利用地图作为长时间记忆，（iii）在分布内和新场景中均优于基于图像的策略，例如在连续操作任务中将成功率提高15%。
英文摘要：
In this paper, we demonstrate that mobile manipulation policies utilizing a 3D latent map achieve stronger spatial and temporal reasoning than policies relying solely on images. We introduce Seeing the Bigger Picture (SBP), an end-to-end policy learning approach that operates directly on a 3D map of latent features. In SBP, the map extends perception beyond the robot's current field of view and aggregates observations over long horizons. Our mapping approach incrementally fuses multiview observations into a grid of scene-specific latent features. A pre-trained, scene-agnostic decoder reconstructs target embeddings from these features and enables online optimization of the map features during task execution. A policy, trainable with behavior cloning or reinforcement learning, treats the latent map as a state variable and uses global context from the map obtained via a 3D feature aggregator. We evaluate SBP on scene-level mobile manipulation and sequential tabletop manipulation tasks. Our experiments demonstrate that SBP (i) reasons globally over the scene, (ii) leverages the map as long-horizon memory, and (iii) outperforms image-based policies in both in-distribution and novel scenes, e.g., improving the success rate by 15% for the sequential manipulation task.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
