LLMs如何计算言语自信
How do LLMs Compute Verbal Confidence
AI总结：
研究探讨了大型语言模型如何内部生成言语自信评分，通过实验发现自信评分在回答生成后被缓存并用于后续输出，揭示了模型自我评估的机制。
摘要：
言语自信——提示LLMs以数字或类别形式陈述其信心——被广泛用于从黑箱模型中提取不确定性估计。然而，LLMs内部如何生成此类评分仍不清楚。我们解答了两个问题：首先，信心是在被请求时即时计算，还是在生成答案时自动计算并缓存以供后续检索；其次，言语自信代表什么——token对数概率，还是更丰富的答案质量评估？我们聚焦于Gemma 3 27B（在TriviaQA、BigMath和MMLU上的表现）、Qwen 2.5 7B以及推理模型Magistral Small 24B，提供了缓存检索的收敛证据。激活引导、修补、噪声和交换实验揭示，信心表示在回答相邻位置先出现，再出现在言语化位置。注意力阻断指出了信息流：信心从回答token中收集，缓存于第一个回答后的位置，然后用于输出。关键发现是线性探测和方差划分揭示，这些缓存表示能够解释超出token对数概率的显著方差，表明是更丰富的答案质量评估，而非简单的流畅性读取。这些发现表明，言语自信反映了自动、复杂的自我评估——而非事后重建——对理解LLMs中的元认知和改进校准具有启示。
英文摘要：
Verbal confidence -- prompting LLMs to state their confidence as a number or category -- is widely used to extract uncertainty estimates from black-box models. However, how LLMs internally generate such scores remains unknown. We address two questions: first, when confidence is computed -- just-in-time when requested, or automatically during answer generation and cached for later retrieval; and second, what verbal confidence represents -- token log-probabilities, or a richer evaluation of answer quality? Focusing on Gemma 3 27B (across TriviaQA, BigMath, and MMLU), Qwen 2.5 7B, and the reasoning model Magistral Small 24B, we provide convergent evidence for cached retrieval. Activation steering, patching, noising, and swap experiments reveal that confidence representations emerge at answer-adjacent positions before appearing at the verbalization site. Attention blocking pinpoints the information flow: confidence is gathered from answer tokens, cached at the first post-answer position, then retrieved for output. Critically, linear probing and variance partitioning reveal that these cached representations explain substantial variance in verbal confidence beyond token log-probabilities, suggesting a richer answer-quality evaluation rather than a simple fluency readout. These findings demonstrate that verbal confidence reflects automatic, sophisticated self-evaluation -- not post-hoc reconstruction -- with implications for understanding metacognition in LLMs and improving calibration.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
