arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.11279cs.CLcs.AI
收藏

多视角一致性增强大型语言模型的置信度估计

Multi-Perspective Consistency Enhances Confidence Estimation in Large Language Models

Meituan(美团);Beijing University of Posts and Telecommunications(北京邮电大学)

Pei Wang, Yejie Wang, Muxi Diao, Keqing He, Guanting Dong, Weiran Xu

AI总结:

针对大型语言模型对错误答案过度自信的问题,本文提出多视角一致性(MPC)方法,利用模型内部和跨模型的互补见解进行缓解,在八个数据集上达到SOTA性能且具备良好扩展性。

摘要:

在大型语言模型(LLMs)的部署中,准确的置信度估计对于评估模型预测的可信度至关重要。然而,现有方法往往无法克服对错误答案过度自信的问题。在本研究中,我们致力于提升大型语言模型的置信度估计。考虑到语言模型自我认知的脆弱性,我们引入了多视角一致性(MPC)方法。我们利用模型内部不同视角(MPC-Internal)和不同模型之间(MPC-Across)的互补见解,来缓解单一视角引发的过度自信问题。在八个公开数据集上的实验结果表明,我们的MPC达到了最先进的性能。进一步分析表明,MPC能够缓解过度自信问题,并且能有效扩展到其他模型。

英文摘要:

In the deployment of large language models (LLMs), accurate confidence estimation is critical for assessing the credibility of model predictions. However, existing methods often fail to overcome the issue of overconfidence on incorrect answers. In this work, we focus on improving the confidence estimation of large language models. Considering the fragility of self-awareness in language models, we introduce a Multi-Perspective Consistency (MPC) method. We leverage complementary insights from different perspectives within models (MPC-Internal) and across different models (MPC-Across) to mitigate the issue of overconfidence arising from a singular viewpoint. The experimental results on eight publicly available datasets show that our MPC achieves state-of-the-art performance. Further analyses indicate that MPC can mitigate the problem of overconfidence and is effectively scalable to other models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑