用于文本引导医学图像分割的定位注入视觉语言语义融合
Localization-Infused Vision-Language Semantic Fusion for Text-Guided Medical Image Segmentation
- Air Force Engineering University（空军工程大学）
- Harbin Institute of Technology（哈尔滨工业大学）
- University of Sydney（悉尼大学）
- Institute of Translational Medicine, Shanghai Jiao Tong University（上海交通大学转化医学研究院）
- Zhongguancun Institute of Artificial Intelligence（中关村人工智能研究院）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
研究针对文本引导医学图像分割，提出LoG框架，通过联合执行多尺度目标定位任务，实现三级定位注入语义融合，在三个基准数据集实验中表现出色，优于现有医学图像分割方法。
AI中文摘要：
医学图像分割对现代计算机辅助医学至关重要。近期，文本引导分割通过纳入临床医生制定的文本报告作为图像分割的语义指导展现出前景。现有方法通常通过预训练文本编码器隐式提取文本语义，再经简单图像 - 文本特征融合整合视觉 - 语言语义。但这些方法未明确捕捉文本报告中面向目标的信息，也未探索基础特征级融合之外的多级信息融合策略。本研究提出LoG框架，通过联合执行多尺度目标定位任务，明确捕捉面向目标的视觉 - 语言语义，实现三级定位注入语义融合，包括定位引导特征融合、定位门控注意力融合和定位约束损失融合。在三个基准数据集上的大量实验表明，LoG在QaTa - COV19、MosMedData + 和Kvasir - SEG上分别取得91.59%、80.71%和94.59%的Dice分数，持续优于现有方法。
英文摘要：
Medical image segmentation is essential for modern computer-aided medicine. Recently, text-guided segmentation has shown promise by incorporating clinician-formulated textual reports as semantic guidance for image segmentation. These textual reports contain language descriptions about the appearance, location, and neighboring anatomy of segmentation targets, providing explicit guidance for target localization and delineation. Existing text-guided segmentation methods typically extract textual semantics implicitly through a pretrained text encoder and then integrate vision-language semantics via straightforward image-text feature fusion. However, these methods do not explicitly capture target-oriented information embedded in textual reports, particularly target location, and do not explore multi-level information fusion strategies beyond basic feature-level fusion, limiting the extraction and integration of critical textual semantics. In this study, we propose LoG, a localization-infused vision-language fusion framework for text-guided medical image segmentation. By jointly performing multi-scale target localization tasks, LoG explicitly captures target-oriented vision-language semantics and enables three-level localization-infused semantic fusion: (i) localization-guided feature fusion that directly infuses location-relevant semantics into visual features, (ii) localization-gated attention fusion that redirects multi-scale localization predictions to reinforce critical regions, and (iii) localization-constrained loss fusion that supervises segmentation based on spatial consistency with target localization. Extensive experiments on three well-established benchmark datasets, involving three medical imaging modalities with paired textual reports, demonstrate that LoG consistently outperforms state-of-the-art medical image segmentation methods.
