arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2212.08228cs.CVcs.AIcs.LG
收藏

SADM:用于纵向医学图像生成的序列感知扩散模型

SADM: Sequence-Aware Diffusion Model for Longitudinal Medical Image Generation

Korea University(高丽大学);The University of British Columbia(不列颠哥伦比亚大学);Columbia University(哥伦比亚大学)

Jee Seok Yoon, Chenghao Zhang, Heung-Il Suk, Jia Guo, Xiaoxiao Li

AI总结:

针对现有医学图像生成方法忽略纵向序列依赖、难以应对序列长度不一、数据缺失及高维等问题,提出引入序列感知Transformer作为条件模块的SADM扩散模型,可在数据缺失下学习纵向依赖并自回归生成图像序列,在3D纵向医学图像实验中效果优于基线方法。

摘要:

由于短期(如心跳)和长期(如衰老)因素的复杂共同作用,人体器官的解剖结构会持续发生变化。显然,在对器官的未来状态建模(即通过图像生成的方式)时,掌握这些因素的先验知识会有所助益。然而,大多数医学图像生成任务仅依赖单张图像的输入,即便存在纵向数据,也忽略了序列间的依赖关系。以有序且带时间戳的图像序列作为模型输入的序列感知深度生成模型,在医学成像领域仍未得到充分研究,该领域面临若干独特挑战:1)序列长度不一;2)数据或帧缺失;3)维度高。为此,我们提出了一种用于纵向医学图像生成的序列感知扩散模型(SADM)。近年来,扩散模型已在高保真图像生成方面展现出可观的效果。我们的方法通过在扩散模型中引入序列感知Transformer作为条件模块,对这项新技术进行了扩展。这一新颖设计使得模型能够在训练过程中即便存在数据缺失的情况下也能学习纵向依赖关系,并在推理过程中支持图像序列的自回归生成。我们在3D纵向医学图像上开展的大量实验表明,与基线方法及其他替代方法相比,SADM具有更优的有效性。代码可在https://github.com/ubc-tea/SADM-Longitudinal-Medical-Image-Generation获取。

英文摘要:

Human organs constantly undergo anatomical changes due to a complex mix of short-term (e.g., heartbeat) and long-term (e.g., aging) factors. Evidently, prior knowledge of these factors will be beneficial when modeling their future state, i.e., via image generation. However, most of the medical image generation tasks only rely on the input from a single image, thus ignoring the sequential dependency even when longitudinal data is available. Sequence-aware deep generative models, where model input is a sequence of ordered and timestamped images, are still underexplored in the medical imaging domain that is featured by several unique challenges: 1) Sequences with various lengths; 2) Missing data or frame, and 3) High dimensionality. To this end, we propose a sequence-aware diffusion model (SADM) for the generation of longitudinal medical images. Recently, diffusion models have shown promising results in high-fidelity image generation. Our method extends this new technique by introducing a sequence-aware transformer as the conditional module in a diffusion model. The novel design enables learning longitudinal dependency even with missing data during training and allows autoregressive generation of a sequence of images during inference. Our extensive experiments on 3D longitudinal medical images demonstrate the effectiveness of SADM compared with baselines and alternative methods. The code is available at https://github.com/ubc-tea/SADM-Longitudinal-Medical-Image-Generation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑