EndoLive：内镜经鼻颅底手术视频的实时风格迁移

中文摘要 AI 辅助

在关键解剖结构周围进行的复杂外科手术，例如内镜经鼻颅底手术，要求外科医生在获准对活体患者进行手术之前进行大量的实践和培训。这种培训通常在尸体标本上进行，因为尸体标本包含与活体人类相同的关键结构。然而，尸体并非活体患者的完美一对一替代品。尸体的死亡和保存组织与活体人类的颜色完全不同，并且——在没有复杂且昂贵的泵系统的情况下——不会以相同的方式出血。因此，在活体病例中识别使该手术如此复杂的关键解剖结构，可能与外科医生在尸体实践中的情况有很大不同。本文提出了EndoLive，一个用于尸体内镜视频和活体人类内镜视频之间实时风格迁移的框架。我们的方法结合了用于外科应用逼真风格迁移的ConStructS GAN模型，以及能够学习复杂转换并实时执行的HyPER-GAN模型。我们在从内窥镜拍摄的未配对尸体和活体图像上训练EndoLive，并在各种设备上使用尸体视频测试训练后的模型。实验结果表明，EndoLive可以以远高于实时所需最低速度的速度进行尸体到活体的转换，同时保持关键解剖结构的语义一致性。我们的源代码可在以下https URL获取。

英文摘要

Complex surgical procedures around critical anatomy, such as the endoscopic endonasal skull base surgery, requires significant practice and training on the part of the surgeon before they are allowed to perform the operation on a live patient. This training in typically done in cadaveric specimens, due to them containing the same critical structures as a living human. However, cadavers are not a perfect 1-to-1 substitute for a living patient. The dead and preserved tissues of a cadaver are colored completely differently than a living human, and -- without complex and expensive pumping systems -- do not bleed in the same way. As a result, identifying the critical pieces of anatomy that make this procedure so complex can be quite different in a live case than in a surgeon's cadaveric practice. This paper presents EndoLive, a framework for real-time style transfer between cadaveric endoscopic video and living human endoscopic video. Our method combines the ConStructS GAN model for realistic style transfer for surgical applications, with the HyPER-GAN model that can learn complex translations and perform them in real-time. We train EndoLive on unpaired cadaveric and live images taken from an endoscope, and test the trained model with cadaveric video, on a variety of devices. Experimental results demonstrate that EndoLive can perform cadaveric-to-live translation at speeds well above the minimum necessary for real-time, while maintaining semantic consistency of critical anatomical structures. Our source code is available at https://github.com/griffhurt/endolive.

发表机构

• University of Pittsburgh（匹兹堡大学）
• University of Pittsburgh School of Computing and Information（匹兹堡大学计算与信息学院）
• School of Computing and Information（计算与信息学院）

机构由 AI 辅助整理，请以论文原文为准。