异构模态学习与抗腐蚀异构模态推断：用于MRI中白质高信号和缺血性卒中病变联合分割
Hetero-modal learning and corruption-resistant hetero-modal inference for joint segmentation of white matter hyperintensities and ischaemic stroke lesions in MRI
- University of Edinburgh（爱丁堡大学）
- Canon Medical Research Europe（佳能医疗研究欧洲）
- UK Dementia Research Institute, Centre at The University of Edinburgh（英国痴呆症研究所爱丁堡中心）
- Usher Institute, University of Edinburgh（爱丁堡大学厄舍研究所）
机构由 AI 辅助整理，请以论文原文为准。
中文总结 AI 辅助
针对WMH和ISL联合分割，提出异构模态学习及抗腐蚀的MMAR模块，在缺失和损坏模态下保持稳健性能。
中文摘要 AI 辅助
白质高信号（WMH）和缺血性卒中病变（ISL）是视觉上相互混淆、常同时出现的病理特征，需要大规模、多样化的数据集才能进行稳健的深度学习分割。然而，由于队列样本缺乏来自两种特征的参考分割以及完整的MRI结构序列（即“模态”）集合，组装此类数据集受到阻碍。为最大化数据利用率，我们研究了异构模态学习，使用包含206名血管疾病患者的数据集，涵盖四种MRI序列（T1加权、T2加权、液体衰减反转恢复和弥散加权成像），并带有WMH和ISL的专家标注。我们证明，在存在大量缺失数据的情况下，异构模态学习优于使用单一成像模态训练的模型，包括一个仅10%的训练数据包含全部四种模态而其余为单模态的分割，以及一个依赖单一共享“锚定”模态且其余模态之间零重叠的分割。此外，在第二种分割下训练的模型成功地对未见过的模态组合进行推断。然而，虽然标准的异构模态网络能够处理缺失序列，但临床部署引入了静默数据退化的额外挑战——即模态存在但严重损坏。为弥合这一差距，我们引入了多模态注意力路由器（MMAR）模块。我们的实验表明，当采用“损坏增强”策略进行训练时，MMAR能有效地动态加权每种模态的编码特征，即使在存在未标记的灾难性损坏模态的情况下，也能在异构模态推断期间保持强劲性能。
英文摘要
White matter hyperintensities (WMH) and ischaemic stroke lesions (ISL) are visually confounding, co-occurring pathologies that require large, diverse datasets for robust deep learning segmentation. However, assembling such datasets is hindered by cohort samples that lack reference segmentations from both features and complete sets of MRI structural sequences (i.e., "modalities"). To maximise data utility, we investigate hetero-modal learning using a dataset of 206 vascular disease patients across four MRI sequences (T1-weighted, T2-weighted, fluid-attenuated inversion recovery, and diffusion-weighted imaging) with expert annotations of both WMH and ISL. We demonstrate that hetero-modal learning outperforms models trained using a single imaging modality in scenarios with substantial missing data, including a split where only 10% of the training data contains all four modalities while the remainder is uni-modal, and a split relying on a single shared "anchor" modality with zero overlap between the remaining modalities. Furthermore, models trained under this second split successfully perform inference on unseen combinations of modalities. Yet, while standard hetero-modal networks handle missing sequences, clinical deployment introduces the additional challenge of silent data degradation - where modalities are present but severely corrupted. To bridge this gap, we introduce the Multimodal Attention Router (MMAR) block. Our experiments demonstrate that, when trained with a "corruption augmentation" strategy, the MMAR effectively dynamically weights the encoded features of each modality, maintaining strong performance during hetero-modal inference even in the presence of unflagged catastrophically corrupted modalities.
