arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2505.09656q-bio.QM
收藏

VIGIL:用于溃疡性结肠炎组织学愈合预测的视觉-语言引导多示例学习框架

VIGIL: Vision-Language Guided Multiple Instance Learning Framework for Ulcerative Colitis Histological Healing Prediction

Zhengxuan Qiu, Bo Peng, Xiaoying Tang, Jiankun Wang, Qin Guo

AI总结:

本文提出VIGIL视觉-语言引导MIL框架,融合WLE与EC并利用报告文本、多层次对齐和MMRF预测UC组织学愈合,以更低标注负担取得92.69%准确率和94.79%AUC。

摘要:

目的:溃疡性结肠炎(UC)以慢性炎症为特征,并伴有缓解与复发交替的周期,需要精确的组织学愈合(HH)评估以改善临床结局。为克服标注密集型深度学习方法的局限,以及多示例学习(MIL)在HH预测中性能欠佳的问题,我们提出VIGIL,这是首个融合白光内镜(WLE)与细胞内镜(EC)的视觉-语言引导MIL框架。方法:VIGIL首先采用基于top-K典型帧选择和相似度度量自适应学习的双分支MIL模块KS-MIL,以有效学习帧特征之间的关系。通过整合诊断报告文本,并在图像-文本对之间设计专门的多层次对齐与监督,VIGIL在训练期间建立联合图像-文本引导,以捕获更丰富的疾病相关语义信息。此外,VIGIL采用多模态掩码关系融合(MMRF)策略,揭示两种内镜图像表示之间潜在的诊断相关性。结果:在真实临床数据集上的综合实验证明了VIGIL的优越性能,其达到92.69%的准确率和94.79%的AUC,优于现有最先进方法。结论:所提出的VIGIL框架成功建立了用于UC HH预测的有效视觉-语言引导MIL范式,在减轻标注负担的同时提高了预测可靠性。意义:研究成果为非侵入性UC诊断提供了新的见解,并对推动智能医疗发展具有理论意义和临床价值。

英文摘要:

Objective: Ulcerative colitis (UC), characterized by chronic inflammation with alternating remission-relapse cycles, requires precise histological healing (HH) evaluation to improve clinical outcomes. To overcome the limitations of annotation-intensive deep learning methods and suboptimal multi-instance learning (MIL) in HH prediction, we propose VIGIL, the first vision-language guided MIL framework integrating white light endoscopy (WLE) and endocytoscopy (EC). Methods:VIGIL begins with a dual-branch MIL module KS-MIL based on top-K typical frames selection and similarity metric adaptive learning to learn relationships among frame features effectively. By integrating the diagnostic report text and specially designed multi-level alignment and supervision between image-text pairs, VIGIL establishes joint image-text guidance during training to capture richer disease-related semantic information. Furthermore, VIGIL employs a multi-modal masked relation fusion (MMRF) strategy to uncover the latent diagnostic correlations of two endoscopic image representations. Results:Comprehensive experiments on a real-world clinical dataset demonstrate VIGIL's superior performance, achieving 92.69\% accuracy and 94.79\% AUC, outperforming existing state-of-the-art methods. Conclusion: The proposed VIGIL framework successfully establishes an effective vision-language guided MIL paradigm for UC HH prediction, reducing annotation burdens while improving prediction reliability. Significance: The research outcomes provide new insights for non-invasive UC diagnosis and hold theoretical significance and clinical value for advancing intelligent healthcare development.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑