A Multimodal Handover Failure Detection Dataset and Baselines
Hochschule Bonn-Rhein-Sieg(波恩-莱茵-锡格应用科学大学);University of Bremen(不莱梅大学);University of Bonn(波恩大学);Lamarr Institute for Machine Learning and Artificial Intelligence(拉马尔人工智能与机器学习研究所)
Santosh Thoduka, Nico Hochgeschwender, Juergen Gall, Paul G. Plöger
An object handover between a robot and a human is a coordinated action which is prone to failure for reasons such as miscommunication, incorrect actions and unexpected object properties. Existing works on handover failure detection and prevention focus on preventing failures due to object slip or external disturbances. However, there is a lack of datasets and evaluation methods that consider unpreventable failures caused by the human participant. To address this deficit, we present the multimodal Handover Failure Detection dataset, which consists of failures induced by the human participant, such as ignoring the robot or not releasing the object. We also present two baseline methods for handover failure detection: (i) a video classification method using 3D CNNs and (ii) a temporal action segmentation approach which jointly classifies the human action, robot action and overall outcome of the action. The results show that video is an important modality, but using force-torque data and gripper position help improve failure detection and action segmentation accuracy.
Benchmarking Quality-Dependent and Cost-Sensitive Score-Level Multimodal Biometric Fusion Algorithms
University of Surrey(萨里大学);West Virginia University(西弗吉尼亚大学);Institut Telecom, Telecom and Management SudParis(法国国立电信学院南巴黎电信与管理学院);CEA LIST(法国原子能与替代能源委员会 LIST 研究所);University of Amsterdam(阿姆斯特丹大学);Brandenburg University of Applied Sciences(勃兰登堡应用技术大学);Otto-von-Guericke-University of Magdeburg(马格德堡奥托·冯·格里克大学)
ERR@HRI 2024 Challenge: Multimodal Detection of Errors and Failures in Human-Robot Interactions
Politecnico di Milano(米兰理工大学);University of Cambridge(剑桥大学);Cornell University(康奈尔大学);Johns Hopkins University(约翰斯·霍普金斯大学);Cankaya University(恰卡亚大学);Cornell Tech(康奈尔科技学院)
Multimodal Coherent Explanation Generation of Robot Failures
Interdepartmental Center for Advances in Robotic Surgery - ICAROS, University of Naples Federico II(跨部门先进机器人手术中心 - ICAROS,那不勒斯费德里科二世大学);Department of Electrical Engineering and Information Technologies - DIETI, University of Naples Federico II(电气工程与信息科技系 - DIETI,那不勒斯费德里科二世大学)
GEM: Glare or Gloom, I Can Still See You -- End-to-End Multimodal Object Detection
Delft University of Technology(代尔夫特理工大学);Czech Institute of Informatics, Robotics, and Cybernetics, Czech Technical University in Prague(布拉格捷克理工大学捷克信息学、机器人与控制论研究所)