可解释多模态AI方法减少不必要的乳腺活检
Explainable Multimodal AI Approach to Reduce Unnecessary Breast Biopsies
AI总结:
本研究提出结合Grad-CAM的多模态视觉-语言模型,在1632名患者的数据集上测试,AUC达0.72,优于单模态基线,可识别高风险患者的良性肿瘤,助力减少不必要的乳腺活检。
摘要:
近期,计算机辅助诊断(CAD)领域的多模态视觉-语言模型有所兴起,但它们在乳腺诊断(尤其是预测活检结果)中的应用仍有限。本研究提出一种多模态视觉-语言方法,结合Grad-CAM作为可解释AI(XAI)技术,用于预测被推荐活检患者的良性肿瘤,尤其是被归类为BI-RADS 4和5类的患者。研究开发了一种新型多模态框架,使用Vision Transformer(ViT)进行影像特征提取,DeBERTa-V3-Large进行文本嵌入。该模型在包含1632名患者的综合内部数据集上进行训练和评估,每名患者有四个乳腺钼靶视图及对应的放射科报告。两种模态的特征被独立提取,在分类前通过后期融合进行整合。Grad-CAM用于生成视觉解释,突出影响模型预测的关键区域。所提模型的灵敏度分别为90%、95%、99%和100%,对应特异度分别为40%、36%、18%和11%。多模态方法在测试集上取得的最佳AUC为0.72,优于仅影像(0.66)和仅文本(0.68)的基线模型。所提结合Grad-CAM的多模态架构,在识别高风险患者中的良性肿瘤方面展现出强大潜力,提供可解释的见解,从而有助于减少不必要的活检、患者焦虑及医疗成本。
英文摘要:
Recently, there has been rise in multimodal vision-language models for computer-aided diagnosis (CAD), yet their application in breast diagnosis, particularly for predicting biopsy outcomes remains limited. This study presents a multimodal vision-language approach, incorporating Grad-CAM as an Explainable AI (XAI) technique, to predict benign tumors in patients recommended for biopsy, especially those classified as BI-RADS 4 and 5. A novel multimodal framework was devel- oped using Vision Transformer (ViT) for imaging feature extraction and DeBERTa-V3-Large for textual embeddings. The model was trained and evaluated on a comprehensive in-house dataset comprising 1,632 patients, each with four mammography views and a corresponding radiology report. The features of both modalities were extracted independently and fused by late fusion before classification. Grad-CAM was used to generate visual explanations highlighting key regions influencing model predictions. The proposed model demonstrated high sensitivity 90%, 95%, 99%, and 100% with corresponding specificities of 40%, 36%, 18%, and 11%, respectively. The best AUC achieved was 0.72 by multimodal approach on the test set, outperforming the image only (0.66) and text-only (0.68) baselines. The proposed multimodal architecture, enhanced with Grad-CAM, demonstrates a strong potential to identify benign tumors among high-risk patients, offering interpretable insights and thereby helping reduce unnecessary biopsies, patient anxiety, and healthcare costs.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。