你到底有多确定？改进医学视觉问答中的口头不确定性校准
Just how sure are you? Improving Verbalized Uncertainty Calibration in Medical VQA
AI总结：
针对多模态大语言模型在医学VQA中过度自信的问题，提出基于复合损失函数的微调框架，通过校准项、锚定正则化、对比对齐和KL稳定项，将校准误差降低60%以上，判别力提升26%以上。
摘要：
应用于医学视觉问答（VQA）的多模态大语言模型（MLLMs）倾向于产生过度自信的输出，无论实际正确性如何，而现有的口头置信度校准方法主要针对纯文本LLMs开发，未考虑医学图像理解的多模态性质。本文提出一个基于训练的框架，通过微调MLLMs来改善其校准，使用复合损失函数，结合Brier风格的校准项、防止置信度向极端值崩溃的锚定正则化、对比图像文本对齐项以及基于KL的模型稳定项。对齐信号来源于一个$2 \ imes 2$因子扰动设计，该设计交叉图像存在性与文本完整性，探测模型对视觉模态输入与语言先验的依赖程度。最后，使用top K KL散度正则化项在微调过程中保护模型的回答能力。在三个医学VQA基准和两种架构（MedGemma 4B IT和Qwen2 VL 7B Instruct）上，我们的方法将校准误差降低了60%以上，判别力提升了26%以上，同时保持了预测准确性。在基准测试的平均结果中，该技术优于基于提示、基于采样和基于训练的方法，消融实验证实损失函数的每个组成部分对于改善校准都是必要的。所有实验代码均已公开。
英文摘要：
Multimodal large language models (MLLMs) applied to Medical Visual Question Answering (VQA) tend to produce overconfident outputs regardless of actual correctness, and existing verbalized confidence calibration methods, developed primarily for text only LLMs, do not account for the multimodal nature of medical image understanding. This work proposes a training based framework that finetunes MLLMs to improve their calibration using a composite loss function combining a Brier style calibration term, an anchor regularizer that prevents confidence collapse toward extreme values, a contrastive image text alignment term, and a KL based model stabilization term. The alignment signal is derived from a $2 \times 2$ factorial perturbation design that crosses image presence with text integrity, probing the reliance of the model on visual modality input versus language priors. Finally, a top K KL divergence regularizer is used to protect the answering ability of the model during finetuning. Across three Medical VQA benchmarks and two architectures (MedGemma 4B IT and Qwen2 VL 7B Instruct), our method reduces calibration error by 60% or more, and improves discrimination by 26% or more, while preserving predictive accuracy. On average across benchmarks, the technique outperforms prompting based, sampling based, and training based approaches, and ablation experiments confirm that each component of the loss function is indeed necessary for improving the calibration. All code for the experiments is publicly available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
