arXiv 2307.100942023-07-19eess.IVcs.CV

Make-A-Volume：利用潜在扩散模型实现跨模态3D脑部MRI合成

Make-A-Volume: Leveraging Latent Diffusion Models for Cross-Modality 3D Brain MRI Synthesis

The University of Hong Kong（香港大学）；The Chinese University of Hong Kong（香港中文大学）；Qilu Hospital of Shandong University（山东大学齐鲁医院）；Nanyang Technological University（南洋理工大学）

Lingting Zhu, Zeyue Xue, Zhenchao Jin, Xian Liu, Jingzhen He, Ziwei Liu, Lequan Yu

备注：: Accepted by International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2023). 10 pages, 4 figures

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对现有医学图像合成方法存在的模式崩溃、训练不稳定及体数据不一致等问题，提出基于2D骨干的扩散框架Make-A-Volume，通过潜在扩散模型和体层微调实现跨模态3D脑部MRI合成，在两个数据集上取得了具备体一致性的优质合成效果。

摘要：

跨模态医学图像合成是一个关键研究主题，有望推动医学成像领域众多应用的发展。尽管近期基于深度学习的生成模型取得了一定进展，但当前大多数医学图像合成方法依赖生成对抗网络，存在众所周知的模式崩溃和训练不稳定问题。此外，基于2D骨干网络的方法容易导致体数据不一致，而3D骨干网络由于内存成本极高、训练难度大，实现起来颇具挑战且不切实际。本文提出一种利用2D骨干网络进行体数据医学图像合成的新范式，并推出基于扩散的框架Make-A-Volume，用于跨模态3D医学图像合成。为学习跨模态的逐切片映射，我们采用潜在扩散模型并学习低维潜在空间，从而实现高计算效率。为实现3D图像合成并缓解体数据不一致问题，我们进一步在2D切片映射模型中插入一系列体层，并使用配对的3D数据对其进行微调。该范式将2D图像扩散模型扩展为体数据版本，仅略微增加参数量和计算量，为通用跨模态3D医学图像合成提供了一套系统性解决方案。我们在内部SWI-MRA脑部MRI数据集和公开的T1-T2脑部MRI数据集上验证了Make-A-Volume框架的有效性。实验结果表明，该框架能够生成具备体一致性的优质合成结果。

英文摘要：

Cross-modality medical image synthesis is a critical topic and has the potential to facilitate numerous applications in the medical imaging field. Despite recent successes in deep-learning-based generative models, most current medical image synthesis methods rely on generative adversarial networks and suffer from notorious mode collapse and unstable training. Moreover, the 2D backbone-driven approaches would easily result in volumetric inconsistency, while 3D backbones are challenging and impractical due to the tremendous memory cost and training difficulty. In this paper, we introduce a new paradigm for volumetric medical data synthesis by leveraging 2D backbones and present a diffusion-based framework, Make-A-Volume, for cross-modality 3D medical image synthesis. To learn the cross-modality slice-wise mapping, we employ a latent diffusion model and learn a low-dimensional latent space, resulting in high computational efficiency. To enable the 3D image synthesis and mitigate volumetric inconsistency, we further insert a series of volumetric layers in the 2D slice-mapping model and fine-tune them with paired 3D data. This paradigm extends the 2D image diffusion model to a volumetric version with a slightly increasing number of parameters and computation, offering a principled solution for generic cross-modality 3D medical image synthesis. We showcase the effectiveness of our Make-A-Volume framework on an in-house SWI-MRA brain MRI dataset and a public T1-T2 brain MRI dataset. Experimental results demonstrate that our framework achieves superior synthesis results with volumetric consistency.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。