知道你所不知道的：视觉语言模型在受损图像上的语言化不确定性估计鲁棒性
Know What You do Not Know: Verbalized Uncertainty Estimation Robustness on Corrupted Images in Vision-Language Models
AI总结：
本文研究视觉语言模型（VLMs）在受损图像上的不确定性估计鲁棒性，发现图像受损严重程度会削弱模型的不确定性估计能力并导致过度自信。
摘要：
为了充分利用大型语言模型（LLMs）的潜力，掌握其答案的不确定性信息至关重要。这意味着模型必须能够量化其对给定响应正确性的确定程度。糟糕的不确定性估计会导致过度自信的错误答案，从而削弱对这些模型的信任。目前针对处理文本输入并输出文本的语言模型已有大量研究。然而，由于视觉能力是最近才加入这些模型的，视觉语言模型（VLMs）的不确定性研究尚未取得太多进展。我们在受损图像数据上测试了三种最先进的VLMs。我们发现，受损严重程度对模型估计其不确定性的能力产生了负面影响，且模型在大多数实验中也表现出过度自信。
英文摘要：
To leverage the full potential of Large Language Models (LLMs) it is crucial to have some information on their answers' uncertainty. This means that the model has to be able to quantify how certain it is in the correctness of a given response. Bad uncertainty estimates can lead to overconfident wrong answers undermining trust in these models. Quite a lot of research has been done on language models that work with text inputs and provide text outputs. Still, since the visual capabilities have been added to these models recently, there has not been much progress on the uncertainty of Visual Language Models (VLMs). We tested three state-of-the-art VLMs on corrupted image data. We found that the severity of the corruption negatively impacted the models' ability to estimate their uncertainty and the models also showed overconfidence in most of the experiments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
