校准带有自生成干扰项的置信度
Calibrating Verbalized Confidence with Self-Generated Distractors
AI总结：
本研究提出DINCO方法，通过自生成干扰项和归一化置信度校准，提升大型语言模型的置信度估计准确性。
摘要：
校准的置信度估计对于大型语言模型（LLM）的输出被人类用户信任是必要的。虽然LLM可以以人类可理解的方式表达其置信度，但LLM生成的口头置信度评分已被实证发现是不准确的，报告在准确性较低的实例上具有高置信度，从而损害信任和安全。我们假设这种过度自信往往源于给定LLM在面对其编码信息较少的声明时的 heightened suggestibility；我们通过实证验证这一假设，发现对低准确性声明的 suggestibility 更高。基于这一发现，我们引入了干扰项归一化一致性（DINCO），通过让模型在多个自生成的干扰项（即替代声明）上独立地口头表达其置信度，并通过总口头置信度进行归一化，来估计并考虑LLM的 suggestibility 偏差。为进一步提高校准，我们利用生成器-验证器的分歧，通过增加归一化验证器置信度与基于一致性的生成器置信度估计来增强。在此，我们将流行的自一致性方法框架为利用采样生成的连贯性，将归一化口头置信度视为利用对不兼容声明的验证上的连贯性，使我们能够将这些互补的连贯性维度整合到DINCO中。此外，我们的分析显示，DINCO提供的置信度估计更不饱和——因此更实用，且进一步采样本身无法缩小DINCO与基线之间的差距，DINCO在10次推理调用中表现优于自一致性在100次调用中的表现。
英文摘要：
Calibrated confidence estimates are necessary for large language model (LLM) outputs to be trusted by human users. While LLMs can express their confidence in human-interpretable ways, verbalized LLM-generated confidence scores have empirically been found to be miscalibrated, reporting high confidence on instances with low accuracy and thereby harming trust and safety. We hypothesize that this overconfidence often stems from a given LLM's heightened suggestibility when faced with claims that it encodes little information about; we empirically validate this hypothesis, finding more suggestibility on lower-accuracy claims. Building on this finding, we introduce Distractor-Normalized Coherence (DINCO), which estimates and accounts for an LLM's suggestibility bias by having the model verbalize its confidence independently across several self-generated distractors (i.e. alternative claims), and normalizes by the total verbalized confidence. To further improve calibration, we leverage generator-validator disagreement, augmenting normalized validator confidence with a consistency-based estimate of generator confidence. Here, we frame the popular approach of self-consistency as leveraging coherence across sampled generations, and normalized verbalized confidence as leveraging coherence across validations on incompatible claims, allowing us to integrate these complementary dimensions of coherence into DINCO. Moreover, our analysis shows that DINCO provides less saturated -- and therefore more usable -- confidence estimates, and that further sampling alone cannot close the gap between DINCO and baselines, with DINCO at 10 inference calls outperforming self-consistency at 100.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
