POV-Surgery：一个用于手术活动期间第一人称手部与工具姿态估计的数据集
POV-Surgery: A Dataset for Egocentric Hand and Tool Pose Estimation During Surgical Activities
AI总结：
针对外科领域第一人称手与工具姿态估计数据集匮乏的问题，研究团队提出了大规模合成数据集POV-Surgery，包含多类手术手套与器械的姿态等标注，验证了现有SOTA方法在真实手术场景的泛化性，且数据集与代码已公开。
摘要：
混合现实（MR）在手术中的应用在手术导航系统、技能评估、机器人辅助手术等领域受到越来越多的关注。对于这类应用而言，从第一人称视角进行手部与手术器械的姿态估计是一项基础任务，近年来在计算机视觉领域得到了广泛研究。然而，该领域的发展受到数据集匮乏的阻碍，尤其是在外科领域，沾血的手套和反光的金属工具使得用传统方法获取手部与物体的3D姿态标注十分困难。为解决这一问题，我们提出了POV-Surgery，这是一个大规模合成第一人称数据集，聚焦于佩戴不同手术手套的手部以及三种骨科手术器械（即手术刀、骨锉（friem）、椎间盘放置器（diskplacer））的姿态估计。我们的数据集包含53个序列、88329帧，具备带活动标注的高分辨率RGB-D视频流、精准的手-物姿态3D与2D标注，以及2D手-物分割掩码。我们在POV-Surgery上对当前的SOTA（ state-of-the-art，最优）方法进行了微调，并通过大量评估进一步验证了其应用于真实场景中带手术手套与工具的情况时的泛化能力。代码与数据集已在batfacewayne.github.io/POV_Surgery_io/ 公开发布。
英文摘要：
The surgical usage of Mixed Reality (MR) has received growing attention in areas such as surgical navigation systems, skill assessment, and robot-assisted surgeries. For such applications, pose estimation for hand and surgical instruments from an egocentric perspective is a fundamental task and has been studied extensively in the computer vision field in recent years. However, the development of this field has been impeded by a lack of datasets, especially in the surgical field, where bloody gloves and reflective metallic tools make it hard to obtain 3D pose annotations for hands and objects using conventional methods. To address this issue, we propose POV-Surgery, a large-scale, synthetic, egocentric dataset focusing on pose estimation for hands with different surgical gloves and three orthopedic surgical instruments, namely scalpel, friem, and diskplacer. Our dataset consists of 53 sequences and 88,329 frames, featuring high-resolution RGB-D video streams with activity annotations, accurate 3D and 2D annotations for hand-object pose, and 2D hand-object segmentation masks. We fine-tune the current SOTA methods on POV-Surgery and further show the generalizability when applying to real-life cases with surgical gloves and tools by extensive evaluations. The code and the dataset are publicly available at batfacewayne.github.io/POV_Surgery_io/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
