arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2210.02685cs.RO
收藏

一种利用神经表面重建实现鲁棒物体抓取的 Real2Sim2Real 方法

A Real2Sim2Real Method for Robust Object Grasping with Neural Surface Reconstruction

Contextual Robotics Institute, UC San Diego(加州大学圣地亚哥分校情境机器人研究所)

Luobin Wang, Runlin Guo, Quan Vuong, Yuzhe Qin, Hao Su, Henrik Christensen

AI总结:

本文提出 Real2Sim2Real 抓取流程,通过神经表面重建将真实点云转为高质量网格并自动生成抓取标签,再训练可迁移至真实场景的鲁棒抓取网络。

摘要:

近期基于 3D 的操作方法要么直接使用 3D 神经网络预测抓取位姿,要么利用从形状数据库中检索到的相似物体求解抓取位姿。然而,前者在使用新机械臂或未见物体进行测试时面临泛化能力挑战;后者则假设数据库中存在相似物体。我们假设,近期的 3D 建模方法为构建评估场景的数字副本提供了一条途径,该副本能够支持物理仿真并支持鲁棒的操作算法学习。我们提出使用最先进的神经表面重建方法,从真实世界点云重建高质量网格(Real2Sim 步骤)。由于大多数仿真器使用网格进行快速仿真,重建的网格能够在无需人工投入的情况下生成抓取位姿标签。生成的标签可训练抓取网络,使其在真实评估场景中表现鲁棒(Sim2Real 步骤)。在合成实验和真实实验中,我们表明 Real2Sim2Real 流程的表现优于使用大型数据集训练的基线抓取网络,以及采用基于检索重建的抓取采样方法。Real2Sim2Real 流程的优势来自:1)将场景建模与抓取采样解耦为子问题;2)两个子问题都可以利用近期的 3D 学习算法和基于网格的物理仿真技术以足够高的质量得到解决。

英文摘要:

Recent 3D-based manipulation methods either directly predict the grasp pose using 3D neural networks, or solve the grasp pose using similar objects retrieved from shape databases. However, the former faces generalizability challenges when testing with new robot arms or unseen objects; and the latter assumes that similar objects exist in the databases. We hypothesize that recent 3D modeling methods provides a path towards building digital replica of the evaluation scene that affords physical simulation and supports robust manipulation algorithm learning. We propose to reconstruct high-quality meshes from real-world point clouds using state-of-the-art neural surface reconstruction method (the Real2Sim step). Because most simulators take meshes for fast simulation, the reconstructed meshes enable grasp pose labels generation without human efforts. The generated labels can train grasp network that performs robustly in the real evaluation scene (the Sim2Real step). In synthetic and real experiments, we show that the Real2Sim2Real pipeline performs better than baseline grasp networks trained with a large dataset and a grasp sampling method with retrieval-based reconstruction. The benefit of the Real2Sim2Real pipeline comes from 1) decoupling scene modeling and grasp sampling into sub-problems, and 2) both sub-problems can be solved with sufficiently high quality using recent 3D learning algorithms and mesh-based physical simulation techniques.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑