arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2502.06233cs.CLcs.AI
收藏

置信度提升 LLM 的自一致性

Confidence Improves Self-Consistency in LLMs

Google Research(谷歌研究);The Hebrew University of Jerusalem(耶路撒冷希伯来大学);Columbia University(哥伦比亚大学)

Amir Taubenfeld, Tom Sheffer, Eran Ofek, Amir Feder, Ariel Goldstein, Zorik Gekhman, Gal Yona

AI总结:

本文提出 CISC,用模型自身置信度对推理路径加权投票,在九模型四数据集上以少逾 40% 的路径平均提升自一致性,并引入问题内置信度评估。

摘要:

自一致性解码通过采样多样化的推理路径并选择最频繁的答案,提升了 LLM 在推理任务上的表现。然而,其计算成本很高,因为需要采样许多此类(冗长的)路径,才能提高正确答案作为最频繁答案出现的概率。为解决这一问题,我们提出了置信度引导自一致性(Confidence-Informed Self-Consistency,CISC)。CISC 基于直接从模型获得的置信度分数执行加权多数投票。通过优先考虑高置信度路径,它能够以显著更小的样本量识别正确答案。在九个模型和四个数据集上测试时,CISC 几乎在所有配置中都优于自一致性,将所需的推理路径数量平均减少了 40% 以上。此外,在证明标准评估方法难以成功区分同一问题的正确与错误答案之后,我们提出了问题内置信度评估的概念。事实上,校准程度最高的置信度方法对 CISC 反而效果最差。最后,除这些实际意义外,我们的结果和分析表明 LLM 能够有效判断自身输出的正确性,从而为围绕这一主题的持续争论提供依据。

英文摘要:

Self-consistency decoding enhances LLMs' performance on reasoning tasks by sampling diverse reasoning paths and selecting the most frequent answer. However, it is computationally expensive, as sampling many of these (lengthy) paths is required to increase the chances that the correct answer emerges as the most frequent one. To address this, we introduce Confidence-Informed Self-Consistency (CISC). CISC performs a weighted majority vote based on confidence scores obtained directly from the model. By prioritizing high-confidence paths, it can identify the correct answer with a significantly smaller sample size. When tested on nine models and four datasets, CISC outperforms self-consistency in nearly all configurations, reducing the required number of reasoning paths by over 40% on average. In addition, we introduce the notion of within-question confidence evaluation, after showing that standard evaluation methods are poor predictors of success in distinguishing correct and incorrect answers to the same question. In fact, the most calibrated confidence method proved to be the least effective for CISC. Lastly, beyond these practical implications, our results and analyses show that LLMs can effectively judge the correctness of their own outputs, contributing to the ongoing debate on this topic.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑