arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.16019cs.CVcs.AI
收藏

MedProbCLIP: 基于概率适应的视觉-语言基础模型用于可靠放射影像-报告检索

MedProbCLIP: Probabilistic Adaptation of Vision-Language Foundation Model for Reliable Radiograph-Report Retrieval

Texas A&M University-San Antonio(德克萨斯A&M大学-圣安东尼奥分校);Boise State University(博伊州立大学)

Ahmad Elallaf, Yu Zhang, Yuktha Priya Masupalli, Jeong Yang, Young Lee, Zechun Cao, Gongbo Liang

AI总结:

MedProbCLIP 通过概率适应提升放射影像与报告检索的可靠性,优于现有基线方法。

摘要:

视觉-语言基础模型已经 emerged 为强大的通用表示学习者,具有强大的多模态理解潜力,但它们的确定性嵌入经常无法提供高风险生物医学应用所需的可靠性。本工作引入了 MedProbCLIP,一种用于胸部X光和放射学报告表示学习及双向检索的概率视觉-语言学习框架。MedProbCLIP 通过概率对比目标将图像和文本表示建模为高斯嵌入,该目标明确捕捉放射影像与临床叙述之间的不确定性和多对多对应关系。变分信息瓶颈缓解了过度自信的预测,而 MedProbCLIP 在训练过程中采用多视图放射影像编码和多部分报告编码,以提供细粒度监督,以实现临床对齐的对应关系,但推理时只需单张放射影像和单份报告。在 MIMIC-CXR 数据集上评估,MedProbCLIP 在检索和零样本分类方面均优于确定性和概率性基线,包括 CLIP、CXR-CLIP 和 PCME++。除了准确性外,MedProbCLIP 还展示了优越的校准、风险覆盖行为、选择性检索可靠性以及对临床相关损坏的鲁棒性,突显了概率视觉-语言建模在提高放射影像-文本检索系统可信度和安全性方面的价值。

英文摘要:

Vision-language foundation models have emerged as powerful general-purpose representation learners with strong potential for multimodal understanding, but their deterministic embeddings often fail to provide the reliability required for high-stakes biomedical applications. This work introduces MedProbCLIP, a probabilistic vision-language learning framework for chest X-ray and radiology report representation learning and bidirectional retrieval. MedProbCLIP models image and text representations as Gaussian embeddings through a probabilistic contrastive objective that explicitly captures uncertainty and many-to-many correspondences between radiographs and clinical narratives. A variational information bottleneck mitigates overconfident predictions, while MedProbCLIP employs multi-view radiograph encoding and multi-section report encoding during training to provide fine-grained supervision for clinically aligned correspondence, yet requires only a single radiograph and a single report at inference. Evaluated on the MIMIC-CXR dataset, MedProbCLIP outperforms deterministic and probabilistic baselines, including CLIP, CXR-CLIP, and PCME++, in both retrieval and zero-shot classification. Beyond accuracy, MedProbCLIP demonstrates superior calibration, risk-coverage behavior, selective retrieval reliability, and robustness to clinically relevant corruptions, underscoring the value of probabilistic vision-language modeling for improving the trustworthiness and safety of radiology image-text retrieval systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑