SpatialPoint: 体感-aware 点预测用于具身定位
SpatialPoint: Spatial-aware Point Prediction for Embodied Localization
AI总结：
本文提出SpatialPoint框架，通过整合结构化深度信息提升具身定位性能，采用260万样本RGB-D数据集进行训练与评估，在真实机器人部署中验证了其在抓取、放置和导航任务中的有效性。
摘要：
具身智能本质上需要在3D空间中确定行动位置的能力。本文将这一需求形式化为具身定位问题，即在视觉观察和语言指令条件下预测可执行的3D点。我们通过两种互补的目标类型实例化具身定位：可触摸点，用于直接物理交互的表面基础3D点；以及空气点，用于指定放置和导航目标、方向约束或几何关系的自由空间3D点。具身定位本质上是具身3D空间推理问题，但现有视觉-语言系统主要依赖RGB输入，需要隐式几何重建，限制了跨场景泛化能力，尽管RGB-D传感器在机器人领域广泛应用。为解决这一差距，我们提出SpatialPoint，一种空间感知的视觉-语言框架，通过精心设计将结构化深度整合到视觉-语言模型（VLM）中，并生成相机坐标系下的3D坐标。我们构建了一个包含260万样本的RGB-D数据集，涵盖可触摸和空气点QA对用于训练和评估。大量实验表明，将深度信息整合到VLMs中显著提高了具身定位性能。我们进一步通过在三个代表性任务上的真实机器人部署验证了SpatialPoint：语言引导的机械臂抓取、物体放置到目标目的地以及移动机器人导航到目标位置。
英文摘要：
Embodied intelligence fundamentally requires a capability to determine where to act in 3D space. We formalize this requirement as embodied localization -- the problem of predicting executable 3D points conditioned on visual observations and language instructions. We instantiate embodied localization with two complementary target types: touchable points, surface-grounded 3D points enabling direct physical interaction, and air points, free-space 3D points specifying placement and navigation goals, directional constraints, or geometric relations. Embodied localization is inherently a problem of embodied 3D spatial reasoning -- yet most existing vision-language systems rely predominantly on RGB inputs, necessitating implicit geometric reconstruction that limits cross-scene generalization, despite the widespread adoption of RGB-D sensors in robotics. To address this gap, we propose SpatialPoint, a spatial-aware vision-language framework with careful design that integrates structured depth into a vision-language model (VLM) and generates camera-frame 3D coordinates. We construct a 2.6M-sample RGB-D dataset covering both touchable and air points QA pairs for training and evaluation. Extensive experiments demonstrate that incorporating depth into VLMs significantly improves embodied localization performance. We further validate SpatialPoint through real-robot deployment across three representative tasks: language-guided robotic arm grasping at specified locations, object placement to target destinations, and mobile robot navigation to goal positions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
