arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2512.24428cs.RO
收藏

亚秒级机器人操作用3D网格生成

Subsecond 3D Mesh Generation for Robot Manipulation

Office of Naval Research(海军研究办公室)

Qian Wang, Omar Abdellall, Tony Gao, Xiatao Sun, Daniel Rakita

AI总结:

本研究提出了一种亚秒级3D网格生成系统,通过集成开放词汇分割、加速扩散生成和稳健点云注册,实现高保真网格的快速生成,用于机器人感知与规划。

摘要:

3D网格是一种广泛应用于计算机科学和工程中的基本表示形式。在机器人领域,它们特别有价值,因为它们以与机器人与物理世界交互方式一致的形式捕捉物体,从而实现预测稳定抓取、检测碰撞和模拟动力学等核心能力。尽管近年来自动3D网格生成方法已取得显著进展, potentially 为实时机器人感知提供了可能的路径,但仍有两个关键挑战。首先,生成高保真的网格对于实时应用来说过于缓慢,通常每个物体需要数十秒。其次,仅靠网格生成本身是不够的。在机器人领域,一个网格必须具有上下文关联性,即正确地从场景中分割并注册到正确的尺度和姿态。此外,除非这些上下文关联步骤保持高效,否则它们仅仅引入了新的瓶颈。在本工作中,我们介绍了一个端到端的系统,解决这些挑战,从单个RGB-D图像在不到一秒内生成高质量的上下文关联3D网格。我们的流程集成了开放词汇物体分割、加速扩散基网格生成和稳健点云注册,每项都针对速度和精度进行了优化。我们通过现实世界操作任务展示了其有效性,证明了它使网格能够作为实用的、按需的表示形式用于机器人感知和规划。

英文摘要:

3D meshes are a fundamental representation widely used in computer science and engineering. In robotics, they are particularly valuable because they capture objects in a form that aligns directly with how robots interact with the physical world, enabling core capabilities such as predicting stable grasps, detecting collisions, and simulating dynamics. Although automatic 3D mesh generation methods have shown promising progress in recent years, potentially offering a path toward real-time robot perception, two critical challenges remain. First, generating high-fidelity meshes is prohibitively slow for real-time use, often requiring tens of seconds per object. Second, mesh generation by itself is insufficient. In robotics, a mesh must be contextually grounded, i.e., correctly segmented from the scene and registered with the proper scale and pose. Additionally, unless these contextual grounding steps remain efficient, they simply introduce new bottlenecks. In this work, we introduce an end-to-end system that addresses these challenges, producing a high-quality, contextually grounded 3D mesh from a single RGB-D image in under one second. Our pipeline integrates open-vocabulary object segmentation, accelerated diffusion-based mesh generation, and robust point cloud registration, each optimized for both speed and accuracy. We demonstrate its effectiveness in a real-world manipulation task, showing that it enables meshes to be used as a practical, on-demand representation for robotics perception and planning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑