arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.08261cs.CVcs.AI
收藏

DBMF：一种用于分布外检测的双分支多模态框架

DBMF: A Dual-Branch Multimodal Framework for Out-of-Distribution Detection

University of Leeds（利兹大学）

Jiangbei Yue, Darren Treanor, Venkataraman Subramanian, Sharib Ali

阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出DBMF框架，通过双分支利用多模态信息提升分布外检测性能，实验表明其在不同模型架构下鲁棒性强，性能提升达24.84%。

摘要：

复杂的现实临床环境需要可靠的深度学习系统。分布外（OOD）检测在遇到偏离训练分布的数据（如未见疾病病例）时，对提高DL模型的可靠性和泛化性至关重要。然而，现有OOD检测方法通常依赖单一视觉模态或仅靠图像-文本匹配，未能充分利用多模态信息。为解决这一挑战，我们提出了一种新的双分支多模态框架，引入文本-图像分支和视觉分支。我们的框架充分利用多模态表示，通过这两个互补分支识别OOD样本。训练后，我们计算文本-图像分支（S_t）和视觉分支（S_v）的分数，并将其整合以获得最终的OOD分数S，该分数与阈值比较以进行OOD检测。在公开可用的内窥镜图像数据集上的全面实验表明，我们提出的框架在多样化的骨干网络上具有鲁棒性，并在OOD检测中将状态-of-the-art性能提高了高达24.84%。

英文摘要：

The complex and dynamic real-world clinical environment demands reliable deep learning (DL) systems. Out-of-distribution (OOD) detection plays a critical role in enhancing the reliability and generalizability of DL models when encountering data that deviate from the training distribution, such as unseen disease cases. However, existing OOD detection methods typically rely either on a single visual modality or solely on image-text matching, failing to fully leverage multimodal information. To overcome the challenge, we propose a novel dual-branch multimodal framework by introducing a text-image branch and a vision branch. Our framework fully exploits multimodal representations to identify OOD samples through these two complementary branches. After training, we compute scores from the text-image branch ($S_t$) and vision branch ($S_v$), and integrate them to obtain the final OOD score $S$ that is compared with a threshold for OOD detection. Comprehensive experiments on publicly available endoscopic image datasets demonstrate that our proposed framework is robust across diverse backbones and improves state-of-the-art performance in OOD detection by up to 24.84%

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑