arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.18319cs.ROcs.CV
收藏

多模态交接失败检测数据集与基线

A Multimodal Handover Failure Detection Dataset and Baselines

Hochschule Bonn-Rhein-Sieg(波恩-莱茵-锡格应用科学大学);University of Bremen(不莱梅大学);University of Bonn(波恩大学);Lamarr Institute for Machine Learning and Artificial Intelligence(拉马尔人工智能与机器学习研究所)

Santosh Thoduka, Nico Hochgeschwender, Juergen Gall, Paul G. Plöger

AI总结:

针对人类引发的不可预防交接失败缺乏数据集和评估方法的问题,提出含人类诱导失败的多模态交接失败检测数据集及两种基线方法,为该领域研究提供数据支撑与基准。

摘要:

机器人与人类之间的物体交接是一种协调动作,易因沟通失误、动作错误和物体特性意外等原因失败。现有交接失败检测与预防研究聚焦于防止物体滑落或外部干扰导致的失败,但缺乏考虑人类参与者引发的不可预防失败的数据集和评估方法。为解决这一不足,我们提出多模态交接失败检测数据集,包含人类引发的失败(如忽视机器人或不松手)。还提出两种基线方法:(i)基于3D CNN的视频分类方法;(ii)联合分类人类动作、机器人动作和整体结果的时序动作分割方法。结果表明视频是重要模态,而使用力-扭矩数据和夹爪位置可提升失败检测与动作分割精度。

英文摘要:

An object handover between a robot and a human is a coordinated action which is prone to failure for reasons such as miscommunication, incorrect actions and unexpected object properties. Existing works on handover failure detection and prevention focus on preventing failures due to object slip or external disturbances. However, there is a lack of datasets and evaluation methods that consider unpreventable failures caused by the human participant. To address this deficit, we present the multimodal Handover Failure Detection dataset, which consists of failures induced by the human participant, such as ignoring the robot or not releasing the object. We also present two baseline methods for handover failure detection: (i) a video classification method using 3D CNNs and (ii) a temporal action segmentation approach which jointly classifies the human action, robot action and overall outcome of the action. The results show that video is an important modality, but using force-torque data and gripper position help improve failure detection and action segmentation accuracy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑