arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.10101cs.RO
收藏

Robo3R:通过精确的前馈3D重建增强机器人操作

Robo3R: Enhancing Robotic Manipulation with Accurate Feed-Forward 3D Reconstruction

Shanghai AI Laboratory(上海人工智能实验室);The Chinese University of Hong Kong(香港中文大学);University of Science and Technology of China(中国科学技术大学);Tsinghua University(清华大学)

Sizhe Yang, Linning Xu, Hao Li, Juncheng Mu, Jia Zeng, Dahua Lin, Jiangmiao Pang

AI总结:

Robo3R通过前馈3D重建模型,直接从RGB图像和机器人状态实时预测高精度场景几何,提升机器人操作的精度与一致性。

摘要:

3D空间感知是通用机器人操作的基础,但获取可靠高质量的3D几何仍具挑战性。深度传感器易受噪声和材料敏感性影响,现有重建模型缺乏物理交互所需的精度和度量一致性。我们引入Robo3R,一种前馈的、适用于操作的3D重建模型,能够从RGB图像和机器人状态实时预测准确的度量级场景几何。Robo3R联合推断尺度不变的局部几何和相对相机姿态,通过学习的全局相似性变换统一到标准机器人框架中。为满足操作的精度需求,Robo3R采用遮罩点头生成锐利的细粒度点云,并基于关键点的透视-n-点(PnP)公式优化相机外参和全局对齐。在Robo3R-4M数据集上训练,Robo3R在多个下游任务中均优于现有重建方法和深度传感器,表明该替代3D传感模块在机器人操作中的潜力。

英文摘要:

3D spatial perception is fundamental to generalizable robotic manipulation, yet obtaining reliable, high-quality 3D geometry remains challenging. Depth sensors suffer from noise and material sensitivity, while existing reconstruction models lack the precision and metric consistency required for physical interaction. We introduce Robo3R, a feed-forward, manipulation-ready 3D reconstruction model that predicts accurate, metric-scale scene geometry directly from RGB images and robot states in real time. Robo3R jointly infers scale-invariant local geometry and relative camera poses, which are unified into the scene representation in the canonical robot frame via a learned global similarity transformation. To meet the precision demands of manipulation, Robo3R employs a masked point head for sharp, fine-grained point clouds, and a keypoint-based Perspective-n-Point (PnP) formulation to refine camera extrinsics and global alignment. Trained on Robo3R-4M, a curated large-scale synthetic dataset with four million high-fidelity annotated frames, Robo3R consistently outperforms state-of-the-art reconstruction methods and depth sensors. Across downstream tasks including imitation learning, sim-to-real transfer, grasp synthesis, and collision-free motion planning, we observe consistent gains in performance, suggesting the promise of this alternative 3D sensing module for robotic manipulation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑