用于可信 PET/CT 肿瘤分割的多模态证据融合网络
Multi-modal Evidential Fusion Network for Trustworthy PET/CT Tumor Segmentation
AI总结：
本文提出多模态证据融合网络 MEFN，通过跨模态特征学习与基于 Dempster-Shafer 理论的可信融合，结合不确定性感知损失，在 PET/CT 肿瘤分割中显著超越现有方法，并能提供可信的分割不确定性。
摘要：
PET/CT 图像中的准确肿瘤分割对于计算机辅助癌症诊断和治疗至关重要。主要挑战在于有效整合来自 PET 和 CT 图像的互补信息。在临床环境中，PET 和 CT 图像的质量往往差异显著，导致网络提取的模态信息存在不确定性。为应对这一挑战，我们提出了一种新颖的多模态证据融合网络（MEFN），该网络包含两个核心阶段：跨模态特征学习（CFL）和多模态可信融合（MTF）。CFL 阶段对不同模态之间的特征进行对齐，并学习更鲁棒的特征表示，从而减轻域差距带来的负面影响。MTF 阶段利用互注意力机制和不确定性校准器，基于模态不确定性对模态特征进行融合，然后在 Dempster-Shafer 理论的指导下融合分割结果。此外，引入了一种新的不确定性感知损失，以迫使模型关注不确定特征，从而提高其提取可信模态信息的能力。我们在两个公开可用的 PET/CT 数据集上进行了广泛的对比实验，以评估所提方法的性能。结果表明，我们的 MEFN 显著优于最先进的方法，在 AutoPET 数据集和 Hecktor 数据集上的 DSC 分数分别提高了 3.10% 和 3.23%。更重要的是，我们的模型能够为放射科医生提供分割结果的可信不确定性，以便他们决定接受或拒绝自动分割结果，这对临床应用尤为重要。我们的代码将在 https://github.com/QPaws/MEFN 上公开。
英文摘要：
Accurate tumor segmentation in PET/CT images is crucial for computer-aided cancer diagnosis and treatment. The primary challenge lies in effectively integrating the complementary information from PET and CT images. In clinical settings, the quality of PET and CT images often varies significantly, leading to uncertainty in the modality information extracted by networks. To address this challenge, we propose a novel Multi-modal Evidential Fusion Network (MEFN), which consists of two core stages: Cross-Modal Feature Learning (CFL) and Multi-modal Trustworthy Fusion (MTF). The CFL stage aligns features across different modalities and learns more robust feature representations, thereby alleviating the negative effects of domain gap. The MTF stage utilizes mutual attention mechanisms and an uncertainty calibrator to fuse modality features based on modality uncertainty and then fuse the segmentation results under the guidance of Dempster-Shafer Theory. Besides, a new uncertainty perceptual loss is introduced to force the model focusing on uncertain features and hence improve its ability to extract trusted modality information. Extensive comparative experiments are conducted on two publicly available PET/CT datasets to evaluate the performance of our proposed method whose results demonstrate that our MEFN significantly outperforms state-of-the-art methods with improvements of 3.10% and 3.23% in DSC scores on the AutoPET dataset and the Hecktor dataset, respectively. More importantly, our model can provide radiologists with credible uncertainty of the segmentation results for their decision in accepting or rejecting the automatic segmentation results, which is particularly important for clinical applications. Our code will be available at https://github.com/QPaws/MEFN.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
