arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.03483cs.CVcs.AI

DuPLUS:面向通用医学图像分割与预后预测的双提示视觉-语言框架

DuPLUS: Dual-Prompt Vision-Language Framework for Universal Medical Image Segmentation and Prognosis

Department of Computer Vision, Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)(计算机视觉系,Mohamed bin Zayed人工智能大学)

Numan Saeed, Tausifa Jan Saleem, Fadillah Maani, Muhammad Ridzuan, Hu Wang, Mohammad Yaqub

AI总结:

针对医学影像专用模型泛化性差、通用方法语义理解弱的问题,提出带双提示分层文本控制架构的DuPLUS框架,在多数据集分割任务上表现优异,还可整合EHR实现预后预测,适配性强。

摘要:

医学影像深度学习领域受限于任务专用模型,这类模型泛化性不足且缺乏预后预测能力,而现有“通用”方法存在条件机制简单、医学语义理解能力薄弱的问题。为解决这些局限,我们提出DuPLUS,一种用于高效多模态医学图像分析的深度学习框架。DuPLUS引入了全新的视觉-语言框架,利用分层语义提示实现对分析任务的细粒度控制,这一能力是此前通用模型所不具备的。为实现向其他医学任务的扩展,该框架包含由独特双提示机制驱动的分层文本控制架构。在分割任务上,DuPLUS可跨3种成像模态、10个解剖结构各异的医学数据集泛化,覆盖30余种器官与肿瘤类型,在10个数据集中的8个上性能优于当前最优的任务专用模型与通用模型。我们通过无缝整合电子健康记录(EHR)数据用于预后预测,验证了其文本控制架构的可扩展性,在头颈癌数据集上,DuPLUS取得了0.69的一致性指数(Concordance Index, CI)。参数高效微调使其能够快速适配来自不同中心的新任务与新模态,确立了DuPLUS作为医学图像分析领域多功能、具备临床相关性解决方案的地位。本研究的代码已公开于:https://anonymous.4open.science/r/DuPLUS-6C52

英文摘要:

Deep learning for medical imaging is hampered by task-specific models that lack generalizability and prognostic capabilities, while existing 'universal' approaches suffer from simplistic conditioning and poor medical semantic understanding. To address these limitations, we introduce DuPLUS, a deep learning framework for efficient multi-modal medical image analysis. DuPLUS introduces a novel vision-language framework that leverages hierarchical semantic prompts for fine-grained control over the analysis task, a capability absent in prior universal models. To enable extensibility to other medical tasks, it includes a hierarchical, text-controlled architecture driven by a unique dual-prompt mechanism. For segmentation, DuPLUS is able to generalize across three imaging modalities, ten different anatomically various medical datasets, encompassing more than 30 organs and tumor types. It outperforms the state-of-the-art task specific and universal models on 8 out of 10 datasets. We demonstrate extensibility of its text-controlled architecture by seamless integration of electronic health record (EHR) data for prognosis prediction, and on a head and neck cancer dataset, DuPLUS achieved a Concordance Index (CI) of 0.69. Parameter-efficient fine-tuning enables rapid adaptation to new tasks and modalities from varying centers, establishing DuPLUS as a versatile and clinically relevant solution for medical image analysis. The code for this work is made available at: https://anonymous.4open.science/r/DuPLUS-6C52

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑