arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.09773cs.CL
收藏

大型语言模型在概率置信度还是语言化置信度方面更诚实?

Are Large Language Models More Honest in Their Probabilistic or Verbalized Confidence?

University of Chinese Academy of Sciences(中国科学院大学)

Shiyu Ni, Keping Bi, Lulu Yu, Jiafeng Guo

AI总结:

本文比较了LLMs对知识边界的概率感知与语言化感知,发现概率感知更准确但需域内验证集调阈值,两者在低频问题上表现更佳,且模型难以用自然语言准确表达内部置信度。

摘要:

大型语言模型(LLMs)在问题超出其内部知识边界时会产生幻觉。一个可靠的模型应清晰感知其知识边界,在知识范围内提供正确答案,并在缺乏知识时拒绝回答。现有关于LLMs知识边界感知的研究通常使用生成标记的概率或语言化置信度作为模型对其响应的置信度。然而,这些研究忽视了这两者之间的差异和联系。在本文中,我们对LLMs对其事实知识边界的概率感知和语言化感知进行了全面分析和比较。首先,我们研究了这两种感知的优缺点。然后,我们研究了它们在频率不同的问题下如何变化。最后,我们衡量了LLMs的概率置信度和语言化置信度之间的相关性。实验结果表明:1)LLMs的概率感知通常比语言化感知更准确,但需要域内验证集来调整置信度阈值。2)两种感知在较少见的问题上表现更好。3)LLMs难以用自然语言准确表达其内部置信度。

英文摘要:

Large language models (LLMs) have been found to produce hallucinations when the question exceeds their internal knowledge boundaries. A reliable model should have a clear perception of its knowledge boundaries, providing correct answers within its scope and refusing to answer when it lacks knowledge. Existing research on LLMs' perception of their knowledge boundaries typically uses either the probability of the generated tokens or the verbalized confidence as the model's confidence in its response. However, these studies overlook the differences and connections between the two. In this paper, we conduct a comprehensive analysis and comparison of LLMs' probabilistic perception and verbalized perception of their factual knowledge boundaries. First, we investigate the pros and cons of these two perceptions. Then, we study how they change under questions of varying frequencies. Finally, we measure the correlation between LLMs' probabilistic confidence and verbalized confidence. Experimental results show that 1) LLMs' probabilistic perception is generally more accurate than verbalized perception but requires an in-domain validation set to adjust the confidence threshold. 2) Both perceptions perform better on less frequent questions. 3) It is challenging for LLMs to accurately express their internal confidence in natural language.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑