arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.08081cs.ROcs.HC
收藏

RoCap:一种用于外观变化物体姿态估计的机器人数据采集流水线

RoCap: A Robotic Data Collection Pipeline for the Pose Estimation of Appearance-Changing Objects

UCLA HCI Research(加州大学洛杉矶分校人机交互研究中心);TOEI Zukun Research(东映未来研究机构);University of Tokyo(东京大学);Future University Hakodate(函馆未来大学)

Jiahao Nick Li, Toby Chong, Zhongyi Zhou, Hironori Yoshida, Koji Yatani, Xiang 'Anthony' Chen, Takeo Igarashi

AI总结:

针对传统视觉姿态估计方法不适用于操作中外观变化物体的问题,提出机器人数据采集流水线Rocap,通过机械臂操作采集带真实6D位姿标注的图像训练模型,效果优于三维重建合成数据训练的模型。

摘要:

在用户将有形物体作为控制器进行操作的混合现实交互中,物体姿态估计发挥着关键作用。传统的基于视觉的物体姿态估计方法利用三维重建来合成训练数据。然而,这些方法是为具有漫反射颜色的静态物体设计的,对于操作过程中外观会发生变化的物体效果不佳,例如毛绒玩具这类可变形物体、化学烧瓶这类透明物体、金属水罐这类反光物体以及剪刀这类铰接式物体。\n为解决这一局限,我们提出了Rocap——一种模拟人类对目标物体进行操作、同时生成带有真实姿态信息标注数据的机器人流水线。用户首先将目标物体交给机械臂,系统会捕捉物体在各种6D位姿配置下的大量图像。系统利用采集到的图像,以及通过机械臂关节角度自动计算得到的真实姿态信息来训练模型。\n我们通过使用采集的数据训练简单的深度学习模型,并将其结果与基于三维重建的合成数据训练的模型进行定量和定性评估,展示了外观变化物体的姿态估计效果。研究结果凸显了Rocap的可观应用潜力。

英文摘要:

Object pose estimation plays a vital role in mixed-reality interactions when users manipulate tangible objects as controllers. Traditional vision-based object pose estimation methods leverage 3D reconstruction to synthesize training data. However, these methods are designed for static objects with diffuse colors and do not work well for objects that change their appearance during manipulation, such as deformable objects like plush toys, transparent objects like chemical flasks, reflective objects like metal pitchers, and articulated objects like scissors. To address this limitation, we propose Rocap, a robotic pipeline that emulates human manipulation of target objects while generating data labeled with ground truth pose information. The user first gives the target object to a robotic arm, and the system captures many pictures of the object in various 6D configurations. The system trains a model by using captured images and their ground truth pose information automatically calculated from the joint angles of the robotic arm. We showcase pose estimation for appearance-changing objects by training simple deep-learning models using the collected data and comparing the results with a model trained with synthetic data based on 3D reconstruction via quantitative and qualitative evaluation. The findings underscore the promising capabilities of Rocap.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑