通过文本和语义定义的分割提示灵活控制3D CT生成
MedSyn2: Flexible Control of 3D CT Generation via Text and Semantically-Defined Segmentation Prompts
AI总结：
提出一种灵活的多模态框架，通过文本和可选分割提示控制3D CT生成，实现高分辨率、解剖一致且可控的体数据生成。
摘要：
体积医学图像的生成模型在医学成像中有许多应用，从数据增强到作为逆问题的先验。对于这些应用，生成具有强可控性的高分辨率3D图像至关重要，但仍极具挑战性。现有方法通常通过放射学报告作为文本提示或通过完整图像分割来控制生成。基于文本的提示虽然灵活，但对异常的位置、形状和边界的空间控制有限。相比之下，基于分割的方法接收精确的空间指导，但需要全器官标注，具有限制性。在这项工作中，我们提出了一种灵活的多模态框架，用于可控体积图像生成，支持来自放射学报告和分割提示（两者均为可选）的输入。我们的方法允许用户提供特定解剖结构或异常的分割，而无需全器官标注。分割掩膜的语义含义通过附带的文本描述指定，从而形成高度灵活且可扩展的条件机制。我们开发了一种基于改进扩散变换器的内存高效架构，该架构联合处理图像和分割标记。该模型进一步结合了门控注意力，以有效关注长放射学报告。实验表明，我们的方法实现了最先进的感知和语义分数（例如，平均FID相对改进24%），生成高分辨率解剖一致的CT体积，并在用于数据增强时提高了数据效率。放射科医生的评估进一步证实了生成图像与真实医学图像之间的强一致性。
英文摘要：
Generative models for volumetric medical images have found many applications in medical imaging, ranging from data augmentation to serving as priors for inverse problems. For these applications, generating high-resolution 3D images with strong controllability is essential but remains highly challenging. Existing approaches typically control generation either through radiology reports used as text prompts or through full image segmentation. While text-based prompting is flexible, it provides limited spatial control over the location, shape, and boundary of abnormalities. In contrast, segmentation-based methods receive precise spatial guidance but are restrictive in requiring full-organ annotations. In this work, we propose a flexible multimodal framework for controllable volumetric image generation that supports input from radiology reports and segmentation prompts (both optional). Our approach allows users to provide segmentation of a specific anatomy or abnormality without requiring full-organ annotations. The semantic meaning of the segmentation mask is specified through an accompanying text description, resulting in a highly flexible and scalable conditioning mechanism. We develop a memory-efficient architecture based on a modified diffusion transformer that jointly processes image and segmentation tokens. The model further incorporates gated attention to effectively attend to long radiology reports. Experiments demonstrate that our method achieves state-of-the-art perceptual and semantic scores (e.g., 24% relative improvement in mean FID), generates high-resolution anatomically consistent CT volumes, and improves data efficiency when used for data augmentation. Radiologists' evaluation further confirms strong alignment between generated and real medical images.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
