通过跨模态协作实现鲁棒的半监督多模态医学图像分割
Robust Semi-supervised Multimodal Medical Image Segmentation via Cross Modality Collaboration
AI总结：
本文提出一种鲁棒的半监督多模态医学图像分割框架，通过跨模态协作蒸馏模态无关知识并结合通道级语义一致性损失与对比一致学习，有效应对标注数据稀缺和模态不对齐问题，在多个医学分割任务中取得竞争性性能。
摘要：
多模态学习利用来自不同模态的互补信息，从而提升医学图像分割的性能。然而，当前主流的多模态学习方法严重依赖来自各种模态的大量高质量标注数据才能实现准确的分割性能。在临床环境中，由于此类数据有限，这种依赖往往构成挑战。此外，不同成像模态之间固有的解剖结构不对齐进一步增加了提升分割性能的难度。为解决这一问题，我们提出了一种新颖的半监督多模态分割框架，该框架对稀缺标注数据和模态不对齐具有鲁棒性。我们的框架采用一种新颖的跨模态协作策略来蒸馏与每个模态固有相关的模态无关知识，并将这些信息整合到一个统一的融合层中进行特征融合。通过通道级语义一致性损失，我们的框架确保跨模态的模态无关信息在特征层面保持对齐，从而增强其在多模态场景下对模态不对齐的抵抗能力。此外，我们的框架有效整合了对比一致学习来约束解剖结构，在半监督分割任务中促进无标注数据上的解剖级预测对齐。在心脏分割、腹部多器官分割和甲状腺相关眼病分割三个任务上，与其他多模态方法相比，我们的方法取得了具有竞争力的性能。同时，在稀缺标注数据和模态不对齐的场景下，它也展现出卓越的鲁棒性。
英文摘要：
Multimodal learning leverages complementary information derived from different modalities, thereby enhancing performance in medical image segmentation. However, prevailing multimodal learning methods heavily rely on extensive well-annotated data from various modalities to achieve accurate segmentation performance. This dependence often poses a challenge in clinical settings due to limited availability of such data. Moreover, the inherent anatomical misalignment between different imaging modalities further complicates the endeavor to enhance segmentation performance. To address this problem, we propose a novel semi-supervised multimodal segmentation framework that is robust to scarce labeled data and misaligned modalities. Our framework employs a novel cross modality collaboration strategy to distill modality-independent knowledge, which is inherently associated with each modality, and integrates this information into a unified fusion layer for feature amalgamation. With a channel-wise semantic consistency loss, our framework ensures alignment of modality-independent information from a feature-wise perspective across modalities, thereby fortifying it against misalignments in multimodal scenarios. Furthermore, our framework effectively integrates contrastive consistent learning to regulate anatomical structures, facilitating anatomical-wise prediction alignment on unlabeled data in semi-supervised segmentation tasks. Our method achieves competitive performance compared to other multimodal methods across three tasks: cardiac, abdominal multi-organ, and thyroid-associated orbitopathy segmentations. It also demonstrates outstanding robustness in scenarios involving scarce labeled data and misaligned modalities.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
