arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.08941cs.LG
收藏

预测熵连接校准与改写敏感性在医疗视觉-语言模型中

Predictive Entropy as a Joint Screen for Error and Paraphrase Instability in Medical Vision-Language Models

SAIL Lab, University of New Haven(纽黑文大学SAIL实验室)

Binesh Sadanandan, Vahid Behzadan

AI总结:

本文研究医疗视觉语言模型在部署中的校准和改写敏感性问题,通过预测熵方法发现决策边界接近性是共同原因,并展示单一熵阈值能有效识别不可靠和改写敏感的预测。

摘要:

医疗视觉语言模型VLMs面临两种威胁安全部署的失败模式:校准不准确的置信度和对问题改写的敏感性。我们通过在MedGemma 4BIT上基准测试五种不确定性量化方法,发现这两种问题共享一个共同原因,即接近决策边界。在跨分布MIMIC CXR和外分布PadChest胸部X光数据集上进行交叉架构验证,使用LLaVA RAD7B。对于校准良好的单模型方法,一次前向传递的预测熵预测哪些样本在改写下会翻转(MedGemma上AUROC为0.711,LLaVA RAD p104上为0.878),使单一熵阈值能够标志不可靠和改写敏感的预测。一个五成员LoRA集合在MIMIC PadChest转移下失败(42.9 ECE,34.1准确率),尽管LLaVA RADs集合不崩溃(69.1)。MC Dropout在5风险下实现最佳校准(ECE 4.3)和选择性预测覆盖率(21.5),但单次前向传递的总熵在错误检测(AUROC 0.743 vs 0.657)和改写筛选中优于集合。简单方法胜出。

英文摘要:

Medical Vision-Language Models (VLMs) answering binary presence questions on chest radiographs can fail in two linked ways: they are confidently wrong, and they change answers when a clinically equivalent question is rephrased. In a binary answer head both failures track the same logit margin, so we test how well one score screens for both. On MedGemma-4B-IT across MIMIC-CXR (in-distribution, n=196) and PadChest (out-of-distribution, n=861), single-pass predictive entropy predicts which yes/no answers flip under rephrasing with an area under the receiver operating characteristic curve (AUROC) of 0.823 on PadChest and 0.821 on MIMIC, and the signal is stable across five random seeds (0.828$\pm$0.021). The same signal replicates on LLaVA-RAD-7B (AUROC 0.83) and is invariant to the flip definition (all, operator-preserving, or negation-excluded rewrites), so the screen needs no semantic filter at inference. One low-entropy threshold drives error and flip rate down together (3.3% and 5.7% at 80% coverage), and threshold selection transfers to a held-out split. Under cross-site shift the more expensive methods we test add nothing practical: at the 5% risk target a single forward pass answers 88.5% of cases, matching Monte Carlo (MC) Dropout, and beats the uniformly averaged five-seed adapter ensemble on error detection by 0.111 AUROC. Even a model that is 91% accurate still contradicts itself on 13% of items under rephrasing, and one forward pass flags them.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑