arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.18130cs.CV

Rad-VLSM:一种结合语义辅助提示的跨模态框架用于医学分割与诊断

Rad-VLSM: A Cross-Modal Framework with Semantics-Assisted Prompting for Medical Segmentation and Diagnosis

Student Member, IEEE(IEEE学生会员);Member, IEEE(IEEE会员)

Fengyi Zhang, Xujie Zeng, Mohan Liu, Zengyi Wang, Yalong Jiang

AI总结:

本文提出Rad-VLSM框架,通过语义引导的提示机制,提升医学图像分割与诊断的准确性,解决现有模型易受背景组织和无关视觉相关性干扰的问题。

摘要:

医学图像分割在支持诊断而非仅仅生成病变掩码时更具临床价值。然而,诊断相关的病变线索往往微妙且局部化,而现有模型可能受背景组织、声学伪影和无关视觉相关性干扰。为了解决这个问题,我们提出了Rad-VLSM,一种两阶段跨模态框架,用于语义辅助的病变聚焦、鲁棒分割和视觉基础诊断。第一阶段中,基于BLIP-2的视觉-语言对齐模块在语义引导下识别病变相关候选区域,并将其转换为框提示。第二阶段中,这些提示被输入基于SAM的多任务网络,其中多候选区域聚合策略提高提示稳定性并引导病变分割。预测的掩码随后用作诊断的空间先验,视觉-放射组学融合头将病变感知的视觉特征与选定的放射组学描述符整合。通过使用语义信息进行定位而非直接预测,Rad-VLSM减少了文本到诊断的依赖,并将诊断基于病变层面的证据。在私有临床乳腺超声数据集和公共基准测试中,Rad-VLSM在分割和诊断性能方面表现强劲,具有良好的泛化能力。

英文摘要:

Medical image segmentation is more clinically valuable when it supports diagnosis rather than merely producing lesion masks. However, diagnostically relevant lesion cues are often subtle and localized, while existing models may be distracted by background tissues, acoustic artifacts, and irrelevant visual correlations. To address this problem, we propose Rad-VLSM, a two-stage cross-modal framework for semantics-assisted lesion focusing, robust segmentation, and visually grounded diagnosis. In the first stage, a BLIP-2-based vision-language alignment module identifies lesion-related candidate regions under semantic guidance and converts them into box prompts. In the second stage, these prompts are fed into a SAM-based multitask network, where a multi-candidate region aggregation strategy improves prompt stability and guides lesion segmentation. The predicted masks are then used as spatial priors for diagnosis, and a visual-radiomics fusion head integrates lesion-aware visual features with selected radiomics descriptors. By using semantic information for localization rather than direct prediction, Rad-VLSM reduces text-to-diagnosis dependence and grounds diagnosis in lesion-level evidence. Experiments on a private clinical breast ultrasound dataset and public benchmarks show that Rad-VLSM achieves strong segmentation and diagnostic performance with favorable generalization.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑