arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2509.05978eess.IVcs.CLcs.CVcs.LG

设想替代方案:通过语言引导实现高分辨率3D反事实医学图像生成

Imagining Alternatives: Towards High-Resolution 3D Counterfactual Medical Image Generation via Language Guidance

McGill University(麦吉尔大学);Mila – Quebec AI Institute(魁北克人工智能研究所)

Mohamed Mohamed, Brennan Nichyporuk, Douglas L. Arnold, Tal Arbel

AI总结:

针对3D医学影像领域缺乏预训练模型、无法仅靠语言生成高分辨率反事实图像的问题,本研究提出结合Simple Diffusion增强的3D扩散框架,首次在神经MRI上实现语言引导的原生3D反事实图像生成,为疾病进展分析奠基。

摘要:

视觉-语言模型已展现出在多种条件下生成2D图像的出色能力;然而,这些模型的成功很大程度上依赖于大量现成的预训练基础模型。关键问题在于,3D领域并不存在可与之相比的预训练模型,这严重限制了相关进展。因此,视觉-语言模型仅基于自然语言生成高分辨率3D反事实医学图像的潜力仍未被探索。填补这一空白将催生强大的临床与研究应用,例如个性化反事实解释、疾病进展模拟,以及通过以真实细节可视化假设病症来优化医学培训。我们的工作向这一挑战迈出了一步,提出了一个能够在自由形式语言提示引导下生成合成患者的高分辨率3D反事实医学图像的框架。我们对最先进的3D扩散模型进行适配,结合Simple Diffusion(简单扩散)的增强机制,并引入增强条件以提升文本对齐效果与图像质量。据我们所知,这是首次将语言引导的原生3D扩散模型应用于神经影像学领域,而在该领域中,精准的三维建模至关重要。在两个神经影像学MRI数据集上,我们的框架模拟了多发性硬化症中不同的反事实病灶负荷,以及阿尔茨海默病中的认知状态,在生成高质量图像的同时保留了受试者特征保真度。我们的研究结果为3D医学影像中基于提示的疾病进展分析奠定了基础。项目链接:https://lesupermomo.github.io/imagining-alternatives/。

英文摘要:

Vision-language models have demonstrated impressive capabilities in generating 2D images under various conditions; however, the success of these models is largely enabled by extensive, readily available pretrained foundation models. Critically, comparable pretrained models do not exist for 3D, significantly limiting progress. As a result, the potential of vision-language models to produce high-resolution 3D counterfactual medical images conditioned solely on natural language remains unexplored. Addressing this gap would enable powerful clinical and research applications, such as personalized counterfactual explanations, simulation of disease progression, and enhanced medical training by visualizing hypothetical conditions in realistic detail. Our work takes a step toward this challenge by introducing a framework capable of generating high-resolution 3D counterfactual medical images of synthesized patients guided by free-form language prompts. We adapt state-of-the-art 3D diffusion models with enhancements from Simple Diffusion and incorporate augmented conditioning to improve text alignment and image quality. To our knowledge, this is the first demonstration of a language-guided native-3D diffusion model applied to neurological imaging, where faithful three-dimensional modeling is essential. On two neurological MRI datasets, our framework simulates varying counterfactual lesion loads in Multiple Sclerosis and cognitive states in Alzheimer's disease, generating high-quality images while preserving subject fidelity. Our results lay the groundwork for prompt-driven disease progression analysis in 3D medical imaging. Project link - https://lesupermomo.github.io/imagining-alternatives/.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑