基于变形测试的手部位姿估计模型鲁棒性评价
Robustness Evaluation in Hand Pose Estimation Models using Metamorphic Testing
摘要:
手部姿态估计(HPE)是从图像或视频帧中预测和描述手部姿态的任务。当HPE模型估计在实验室或受控环境中捕获的手部姿势时,它们通常会提供良好的性能。然而,现实世界的环境是复杂的,可能会发生各种不确定性,这可能会降低HPE模型的性能。例如,手可能被遮挡,手的可见性可能由于不完美的曝光率而降低,并且手的轮廓在快速手运动期间易于模糊。在这项工作中,我们采用变形测试来评估HPE模型的鲁棒性,并为不同的应用提供HPE模型选择的建议。鲁棒性评估是在四种最先进的模型上进行的,即MediaPipe手部、OpenPose、BodyHands和NSRM手部。我们发现,当引入对角线运动模糊时,平均超过80%的手不能被BodyHands识别,至少50%的手不能被MediaPipe手识别,而平均超过50%的强烈曝光不足的手不能被NSRM手正确地估计。类似地,仅在四个手部关节上应用遮挡也会大大降低这些模型的性能。实验结果表明,遮挡,光照变化和运动模糊是现有HPE模型性能的主要障碍。这些发现可能为研究人员改善手部姿态估计模型及其应用的性能和鲁棒性铺平了道路。
英文摘要:
Hand pose estimation (HPE) is a task that predicts and describes the hand poses from images or video frames. When HPE models estimate hand poses captured in a laboratory or under controlled environments, they normally deliver good performance. However, the real-world environment is complex, and various uncertainties may happen, which could degrade the performance of HPE models. For example, the hands could be occluded, the visibility of hands could be reduced by imperfect exposure rate, and the contour of hands prone to be blurred during fast hand movements. In this work, we adopt metamorphic testing to evaluate the robustness of HPE models and provide suggestions on the choice of HPE models for different applications. The robustness evaluation was conducted on four state-of-the-art models, namely MediaPipe hands, OpenPose, BodyHands, and NSRM hand. We found that on average more than 80\% of the hands could not be identified by BodyHands, and at least 50\% of hands could not be identified by MediaPipe hands when diagonal motion blur is introduced, while an average of more than 50\% of strongly underexposed hands could not be correctly estimated by NSRM hand. Similarly, applying occlusions on only four hand joints will also largely degrade the performance of these models. The experimental results show that occlusions, illumination variations, and motion blur are the main obstacles to the performance of existing HPE models. These findings may pave the way for researchers to improve the performance and robustness of hand pose estimation models and their applications.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。