arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.35704cs.AI
收藏

可解释多模态AI方法减少不必要的乳腺活检

Explainable Multimodal AI Approach to Reduce Unnecessary Breast Biopsies

School of Engineering and Sciences, Tecnologico de Monterrey(蒙特雷理工学院工程与科学学院);School of Medicine and Health Sciences, Tecnologico de Monterrey(蒙特雷理工学院医学与健康科学学院)

Sadam Hussain, Mansoor Ali, Daly Betzabeth Avendaño-Avalos, Jose Gerardo Tamez-Peña

AI总结:

本研究提出结合Grad-CAM的多模态视觉-语言模型,在1632名患者的数据集上测试,AUC达0.72,优于单模态基线,可识别高风险患者的良性肿瘤,助力减少不必要的乳腺活检。

摘要:

近期,计算机辅助诊断(CAD)领域的多模态视觉-语言模型有所兴起,但它们在乳腺诊断(尤其是预测活检结果)中的应用仍有限。本研究提出一种多模态视觉-语言方法,结合Grad-CAM作为可解释AI(XAI)技术,用于预测被推荐活检患者的良性肿瘤,尤其是被归类为BI-RADS 4和5类的患者。研究开发了一种新型多模态框架,使用Vision Transformer(ViT)进行影像特征提取,DeBERTa-V3-Large进行文本嵌入。该模型在包含1632名患者的综合内部数据集上进行训练和评估,每名患者有四个乳腺钼靶视图及对应的放射科报告。两种模态的特征被独立提取,在分类前通过后期融合进行整合。Grad-CAM用于生成视觉解释,突出影响模型预测的关键区域。所提模型的灵敏度分别为90%、95%、99%和100%,对应特异度分别为40%、36%、18%和11%。多模态方法在测试集上取得的最佳AUC为0.72,优于仅影像(0.66)和仅文本(0.68)的基线模型。所提结合Grad-CAM的多模态架构,在识别高风险患者中的良性肿瘤方面展现出强大潜力,提供可解释的见解,从而有助于减少不必要的活检、患者焦虑及医疗成本。

英文摘要:

Recently, there has been rise in multimodal vision-language models for computer-aided diagnosis (CAD), yet their application in breast diagnosis, particularly for predicting biopsy outcomes remains limited. This study presents a multimodal vision-language approach, incorporating Grad-CAM as an Explainable AI (XAI) technique, to predict benign tumors in patients recommended for biopsy, especially those classified as BI-RADS 4 and 5. A novel multimodal framework was devel- oped using Vision Transformer (ViT) for imaging feature extraction and DeBERTa-V3-Large for textual embeddings. The model was trained and evaluated on a comprehensive in-house dataset comprising 1,632 patients, each with four mammography views and a corresponding radiology report. The features of both modalities were extracted independently and fused by late fusion before classification. Grad-CAM was used to generate visual explanations highlighting key regions influencing model predictions. The proposed model demonstrated high sensitivity 90%, 95%, 99%, and 100% with corresponding specificities of 40%, 36%, 18%, and 11%, respectively. The best AUC achieved was 0.72 by multimodal approach on the test set, outperforming the image only (0.66) and text-only (0.68) baselines. The proposed multimodal architecture, enhanced with Grad-CAM, demonstrates a strong potential to identify benign tumors among high-risk patients, offering interpretable insights and thereby helping reduce unnecessary biopsies, patient anxiety, and healthcare costs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑