单目腹腔镜视频片段中解剖结构与手术器械三维跟踪的神经场
Neural Fields for 3D Tracking of Anatomy and Surgical Instruments in Monocular Laparoscopic Video Clips
AI总结：
本文提出基于单目腹腔镜视频训练神经场以联合跟踪解剖结构与手术器械，并通过增强类别加权提升小尺寸器械跟踪精度，验证了神经场用于腹腔镜单目三维重建的可行性。
摘要：
腹腔镜视频跟踪主要关注两类目标：手术器械和解剖结构。前者可用于技能评估，而后者是虚拟叠加投影所必需的。器械跟踪和解剖结构跟踪通常被视为两个独立的问题，本文提出了一种同时联合跟踪所有结构的方法。基于单个二维单目视频片段，我们训练一个神经场来表示连续的时空场景，用于生成至少在某一帧中可见的所有表面的三维轨迹。由于器械尺寸较小，通常只覆盖图像的一小部分，导致跟踪精度下降。因此，我们提出增强类别加权以改善器械轨迹。我们在腹腔镜胆囊切除术的视频片段上评估跟踪性能，发现解剖结构的平均跟踪精度为92.4%，器械的平均跟踪精度为87.4%。此外，我们评估了由该方法场景重建获得的深度图质量。我们表明，这些伪深度具有与最先进的预训练深度估计器相当的质量。在SCARED数据集的腹腔镜视频上，该方法预测深度的平均绝对误差为2.9毫米，相对误差为9.2%。这些结果表明，使用神经场进行腹腔镜场景的单目三维重建是可行的。
英文摘要：
Laparoscopic video tracking primarily focuses on two target types: surgical instruments and anatomy. The former could be used for skill assessment, while the latter is necessary for the projection of virtual overlays. Where instrument and anatomy tracking have often been considered two separate problems, in this paper, we propose a method for joint tracking of all structures simultaneously. Based on a single 2D monocular video clip, we train a neural field to represent a continuous spatiotemporal scene, used to create 3D tracks of all surfaces visible in at least one frame. Due to the small size of instruments, they generally cover a small part of the image only, resulting in decreased tracking accuracy. Therefore, we propose enhanced class weighting to improve the instrument tracks. We evaluate tracking on video clips from laparoscopic cholecystectomies, where we find mean tracking accuracies of 92.4% for anatomical structures and 87.4% for instruments. Additionally, we assess the quality of depth maps obtained from the method's scene reconstructions. We show that these pseudo-depths have comparable quality to a state-of-the-art pre-trained depth estimator. On laparoscopic videos in the SCARED dataset, the method predicts depth with an MAE of 2.9 mm and a relative error of 9.2%. These results show the feasibility of using neural fields for monocular 3D reconstruction of laparoscopic scenes.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
