过度自信与校准在医学视觉问答中的研究：实证发现与幻觉意识缓解
Overconfidence and Calibration in Medical VQA: Empirical Findings and Hallucination-Aware Mitigation
AI总结：
本文研究了医学VQA中VLMs的过度自信问题，发现校准方法如Platt缩放能有效提升校准精度，但无法提升预测区分度，提出幻觉意识校准方法以提高可靠性。
摘要：
随着视觉语言模型（VLMs）在临床决策支持中的广泛应用，准确率之外，了解何时信任其预测同样关键。然而，医学领域对这些模型过度自信的系统研究仍显不足。本文通过跨三个模型家族（Qwen3-VL、InternVL3、LLaVA-NeXT）、三个模型规模（2B-38B）和多种置信度估计提示策略的综合实证研究，揭示了三个关键发现：首先，过度自信在不同模型家族中持续存在，且无法通过规模或提示解决；其次，简单的后验校准方法如Platt缩放能减少校准误差并优于提示策略；第三，由于这些后验校准方法的严格单调性，它们在提升预测区分度方面有限，使AUROC保持不变。受此启发，本文探讨了幻觉意识校准（HAC），通过将基于视觉的幻觉检测信号作为补充输入来优化置信度估计。结果显示，利用这些幻觉信号可同时提升校准和AUROC，尤其在开放性问题中效果显著。总体而言，本文发现后验校准应作为医学VLM部署的标准实践，以替代原始置信度估计，并强调幻觉信号在医疗VQA中应用的实用性。
英文摘要：
As vision-language models (VLMs) are increasingly deployed in clinical decision support, more than accuracy is required: knowing when to trust their predictions is equally critical. Yet, a comprehensive and systematic investigation into the overconfidence of these models remains notably scarce in the medical domain. We address this gap through a comprehensive empirical study of confidence calibration in VLMs, spanning three model families (Qwen3-VL, InternVL3, LLaVA-NeXT), three model scales (2B--38B), and multiple confidence estimation prompting strategies, across three medical visual question answering (VQA) benchmarks. Our study yields three key findings: First, overconfidence persists across model families and is not resolved by scaling or prompting, such as chain-of-thought and verbalized confidence variants. Second, simple post-hoc calibration approaches, such as Platt scaling, reduce calibration error and consistently outperform the prompt-based strategy. Third, due to their (strict) monotonicity, these post-hoc calibration methods are inherently limited in improving the discriminative quality of predictions, leaving AUROC at the same level. Motivated by these findings, we investigate hallucination-aware calibration (HAC), which incorporates vision-grounded hallucination detection signals as complementary inputs to refine confidence estimates. We find that leveraging these hallucination signals improves both calibration and AUROC, with the largest gains on open-ended questions. On closed-ended questions, format-specific signals such as the logit margin show more informative. Overall, our findings suggest post-hoc calibration as standard practice for medical VLM deployment over raw confidence estimates, and highlight the practical usefulness of hallucination signals to enable more reliable use of VLMs in medical VQA.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
