早期至晚期融合对来自未完美配准多模态MRI的胰腺分割的影响
Influence of Early through Late Fusion on Pancreas Segmentation from Imperfectly Registered Multimodal MRI
AI总结：
针对未完美配准的T2w/T1w腹部MRI，本文比较UNet与nnUNet中从早期到晚期的融合位置，发现最佳融合点具有模型特异性且提升有限。
摘要：
多模态融合有望改善胰腺分割。然而，在模型中何处执行融合仍是一个悬而未决的问题。在分析未完美对齐的图像对时，尚不清楚是否存在融合信息的最佳位置。本胰腺分割研究中的两个主要对齐挑战是：1）胰腺是可变形的，2）呼吸会使腹部变形。即使在图像配准之后，相关变形通常也未得到校正。我们研究了从早期到晚期的融合如何影响胰腺分割。我们使用了来自163名受试者的353对T2加权（T2w）和T1加权（T1w）腹部MR图像，并附有胰腺标签。我们使用图像配准（deeds）来对齐图像对。我们训练了一组具有不同融合点（从早期到晚期）的基础UNet，以评估在未完美对齐的图像上，从早期到晚期的融合如何影响分割性能。我们在nnUNet上评估了融合点的泛化能力。使用基础UNet模型的单模态T2w基线的Dice得分为0.73，而nnUNet模型上的同一基线达到0.80。对于基础UNet，最佳融合方法出现在编码器中部（早期/中期融合），与基线相比，其Dice得分在统计上显著提高了0.0125。对于nnUNet，最佳融合方法是模型前的朴素图像拼接（早期融合），与基线相比，其Dice得分在统计上显著提高了0.0021。在特定块中进行融合可以提高性能，但最佳融合块因模型而异，且增益较小。在未完美配准的数据集中，融合是一个微妙的问题，设计技巧对于发掘潜在见解仍然至关重要。未来需要创新，以更好地解决腹部图像对未完美对齐情况下的融合问题。
英文摘要：
Multimodal fusion promises better pancreas segmentation. However, where to perform fusion in models is still an open question. It is unclear if there is a best location to fuse information when analyzing pairs of imperfectly aligned images. Two main alignment challenges in this pancreas segmentation study are 1) the pancreas is deformable and 2) breathing deforms the abdomen. Even after image registration, relevant deformations are often not corrected. We examine how early through late fusion impacts pancreas segmentation. We used 353 pairs of T2-weighted (T2w) and T1-weighted (T1w) abdominal MR images from 163 subjects with accompanying pancreas labels. We used image registration (deeds) to align the image pairs. We trained a collection of basic UNets with different fusion points, spanning from early to late, to assess how early through late fusion influenced segmentation performance on imperfectly aligned images. We assessed generalization of fusion points on nnUNet. The single-modality T2w baseline using a basic UNet model had a Dice score of 0.73, while the same baseline on the nnUNet model achieved 0.80. For the basic UNet, the best fusion approach occurred in the middle of the encoder (early/mid fusion), which led to a statistically significant improvement of 0.0125 on Dice score compared to the baseline. For the nnUNet, the best fusion approach was naïve image concatenation before the model (early fusion), which resulted in a statistically significant Dice score increase of 0.0021 compared to baseline. Fusion in specific blocks can improve performance, but the best blocks for fusion are model specific, and the gains are small. In imperfectly registered datasets, fusion is a nuanced problem, with the art of design remaining vital for uncovering potential insights. Future innovation is needed to better address fusion in cases of imperfect alignment of abdominal image pairs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
