arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.03992eess.IV
收藏

面向高级分析的医学图像融合:一种用于未配准PAT与MRI的互增强框架

Medical Image Fusion for High-Level Analysis: A Mutual Enhancement Framework for Unaligned PAT and MRI

Yutian Zhong, Jinchuan He, Zhichao Liang, Shuangyang Zhang, Qianjin Feng, Lijun Lu, Li Qi

AI总结:

针对现有医学图像融合方法依赖预配准、忽略高级任务需求的问题,提出整合生成与配准的无监督融合模型PAMRFuse+,在未配准PAT-MRI数据上取得良好效果,并首次用多器官实例分割验证融合对高级分析的提升作用。

摘要:

光声断层成像(PAT)可提供光学对比度,而磁共振成像(MRI)在软组织与器官解剖结构成像方面表现优异。由于二者优势互补,PAT与MRI的融合具备可观的应用前景。现有图像融合方法在预配准图像上已取得长足进展,但医学成像场景中空间形变难以避免。更关键的是,当前算法聚焦于视觉质量与统计指标,因而忽略了高级任务的需求。为应对这些挑战,我们提出一种无监督融合模型,名为PAMRFuse+,该模型整合了图像生成与配准功能。具体而言,我们引入跨模态风格迁移网络,将跨模态配准简化为单模态配准;随后采用多级配准网络预测位移矢量场。此外,我们提出双分支特征分解融合网络,通过整合模态特异性与模态共享特征,解决跨模态特征建模与分解的难题。PAMRFuse+在未配准PAT-MRI数据集的配准与融合任务中取得了令人满意的结果。此外,我们首次采用多器官实例分割评估医学图像融合的性能。大量实验验证表明,PAMRFuse+在提升医学图像分析任务性能方面具备优势。

英文摘要:

Photoacoustic tomography (PAT) offers optical contrast, whereas magnetic resonance imaging (MRI) excels in imaging soft tissue and organ anatomy. The fusion of PAT with MRI holds promising application prospects due to their complementary advantages. Existing image fusion have made considerable progress in pre-registered images, yet spatial deformations are difficult to avoid in medical imaging scenarios. More importantly, current algorithms focus on visual quality and statistical metrics, thus overlooking the requirements of high-level tasks. To address these challenges, we propose an unsupervised fusion model, termed PAMRFuse+, which integrates image generation and registration. Specifically, a cross-modal style transfer network is introduced to simplify cross-modal registration to single-modal registration. Subsequently, a multi-level registration network is employed to predict displacement vector fields. Furthermore, a dual-branch feature decomposition fusion network is proposed to address the challenges of cross-modal feature modeling and decomposition by integrating modality-specific and modality-shared features. PAMRFuse+ achieves satisfactory results in registering and fusing unaligned PAT-MRI datasets. Moreover, for the first time, we evaluate the performance of medical image fusion with multi-organ instance segmentation. Extensive experimental demonstrations reveal the advantages of PAMRFuse+ in improving the performance of medical image analysis tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑