一种用于外科成像中高光谱去马赛克与RGB重建的自监督和对抗方法
A self-supervised and adversarial approach to hyperspectral demosaicking and RGB reconstruction in surgical imaging
AI总结：
提出一种自监督对抗学习方法，用于外科成像中的高光谱去马赛克与RGB重建，无需配对高分辨率数据，在空间和光谱保真度及视觉评估上优于现有方法。
摘要：
高光谱成像在外科成像中具有广阔前景，能够提供肉眼不可见的生物组织区分能力和详细信息。对于术中导航，必须实时捕获和显示光谱数据。鉴于这一要求，快照马赛克高光谱相机目前被视为最合适的技术。然而，快照马赛克成像需要去马赛克算法来完全恢复图像中的空间和光谱细节。现代去马赛克方法通常依赖合成数据集来开发监督学习方法，因为在实际中几乎不可能同时捕获完全相同外科场景的快照和高分辨率光谱图像。在本研究中，我们提出了一种自监督去马赛克和RGB重建方法，该方法不依赖配对的高分辨率数据作为真实基准。我们利用未配对的标准高分辨率外科显微镜图像，这些图像仅提供RGB数据，但可以在常规手术中收集。通过对抗学习辅以自监督方法，我们驱动基于高光谱的RGB重建，使其类似于外科显微镜图像，并提高去马赛克的空间分辨率。重建的高光谱图像的空间和光谱保真度已进行定量评估。此外，还进行了一项用户研究，以评估从这些光谱图像生成的RGB可视化效果。空间细节和颜色准确性均由神经外科专家进行评估。我们提出的自监督去马赛克方法与现有方法相比显示出改进的结果，证明了其无缝集成到术中工作流程中的潜力。
英文摘要：
Hyperspectral imaging holds promises in surgical imaging by offering biological tissue differentiation capabilities with detailed information that is invisible to the naked eye. For intra-operative guidance, real-time spectral data capture and display is mandated. Snapshot mosaic hyperspectral cameras are currently seen as the most suitable technology given this requirement. However, snapshot mosaic imaging requires a demosaicking algorithm to fully restore the spatial and spectral details in the images. Modern demosaicking approaches typically rely on synthetic datasets to develop supervised learning methods, as it is practically impossible to simultaneously capture both snapshot and high-resolution spectral images of the exact same surgical scene. In this work, we present a self-supervised demosaicking and RGB reconstruction method that does not depend on paired high-resolution data as ground truth. We leverage unpaired standard high-resolution surgical microscopy images, which only provide RGB data but can be collected during routine surgeries. Adversarial learning complemented by self-supervised approaches are used to drive our hyperspectral-based RGB reconstruction into resembling surgical microscopy images and increasing the spatial resolution of our demosaicking. The spatial and spectral fidelity of the reconstructed hyperspectral images have been evaluated quantitatively. Moreover, a user study was conducted to evaluate the RGB visualisation generated from these spectral images. Both spatial detail and colour accuracy were assessed by neurosurgical experts. Our proposed self-supervised demosaicking method demonstrates improved results compared to existing methods, demonstrating its potential for seamless integration into intra-operative workflows.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
