arXiv 2603.105192026-03-11cs.CV

通过细粒度语义解耦实现视觉引导的可控医学图像生成

Visually-Guided Controllable Medical Image Generation via Fine-Grained Semantic Disentanglement

Computer Science and Engineering, Northeastern University, Shenyang, China（东北大学计算机科学与工程系，中国沈阳）；Key Laboratory of Intelligent Computing in Medical Image of Ministry of Education, Northeastern University, Shenyang, China（教育部医学图像智能计算重点实验室，东北大学，中国沈阳）；National Frontiers Science Center for Industrial Intelligence and Systems Optimization, Shenyang, China（工业智能与系统优化国家级前沿科学中心，中国沈阳）；AiShiWeiLai AI Research, China（艾世维来人工智能研究，中国）；Amii, University of Alberta, Edmonton, Alberta, Canada（阿尔伯塔大学艾米人工智能研究所，加拿大埃德蒙顿，阿尔伯塔）

Xin Huang, Junjie Liang, Qingshan Hou, Peng Cao, Jinzhu Yang, Xiaoli Liu, Osmar R. Zaiane

备注：: 10 pages, 7 figures. Currently under review

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出视觉引导的文本解耦框架，通过细粒度语义解耦提升医学图像生成的可控性和生成质量。

摘要：

医学图像合成对于缓解数据稀缺和隐私约束至关重要。然而，微调通用文本到图像（T2I）模型仍然具有挑战性，主要原因是复杂视觉细节与抽象临床文本之间存在显著的模态差距。此外，语义纠缠仍然存在，其中粗粒度文本嵌入模糊了解剖结构和成像风格之间的边界，从而在生成过程中削弱了可控性。为了解决这一问题，我们提出了一种视觉引导的文本解耦框架。我们引入了一种跨模态潜在对齐机制，利用视觉先验来明确将无结构文本分解为独立的语义表示。随后，混合特征融合模块（HFFM）将这些特征通过分离的通道注入到扩散变换器（DiT）中，从而实现细粒度的结构控制。在三个数据集上的实验结果表明，我们的方法在生成质量上优于现有方法，并在下游分类任务上显著提高了性能。源代码可在https://github.com/hx111/VG-MedGen上获得。

英文摘要：

Medical image synthesis is crucial for alleviating data scarcity and privacy constraints. However, fine-tuning general text-to-image (T2I) models remains challenging, mainly due to the significant modality gap between complex visual details and abstract clinical text. In addition, semantic entanglement persists, where coarse-grained text embeddings blur the boundary between anatomical structures and imaging styles, thus weakening controllability during generation. To address this, we propose a Visually-Guided Text Disentanglement framework. We introduce a cross-modal latent alignment mechanism that leverages visual priors to explicitly disentangle unstructured text into independent semantic representations. Subsequently, a Hybrid Feature Fusion Module (HFFM) injects these features into a Diffusion Transformer (DiT) through separated channels, enabling fine-grained structural control. Experimental results in three datasets demonstrate that our method outperforms existing approaches in terms of generation quality and significantly improves performance on downstream classification tasks. The source code is available at https://github.com/hx111/VG-MedGen.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。