LLM中的报告置信度追踪承诺而非正确性
Reported Confidence in LLMs Tracks Commitment More Than Correctness
AI总结：
本研究通过两阶段弃权范式，发现LLM的言语置信度预测弃权决策远优于预测答案正确性，而令牌对数概率则相反，表明言语置信度是内部承诺准备状态的行为输出。
摘要：
置信度是对所选答案正确概率的估计。言语置信度报告被广泛用作大语言模型的不确定性度量，但它们是否最好被理解为正确性的估计尚不清楚。我们使用来自感知决策神经科学的两阶段弃权范式进行测试：模型首先回答并报告其置信度，然后决定是将其提交给用户还是弃权。在四个非推理模型、多种提示框架和置信度格式中，言语置信度预测弃权/提交决策的能力显著优于预测答案是否正确。校准后的令牌对数概率显示出相反的特征，弃权预测与正确性判别相关联，这是答案证据信号的标志。在移除言语置信度与对数概率共享的方差后，残差仍与承诺保持一致，而它与正确性的联系降至接近随机水平。这种分离在四个推理模型中泛化，涵盖从困难多项选择到前沿自由形式问题的四个不同难度基准。在Gemma 3和4中的机制分析是一致的：已知因果支持言语置信度生成的答案后状态，在弃权提示之前已经编码了未来的弃权决策，主要按该决策而非正确性组织，两者在激活空间中大致正交。沿着言语置信度特定方向进行干预因果地改变了弃权行为。因此，言语置信度和对数概率置信度不可互换：对数概率跟踪答案证据和正确性，而言语置信度则更好地理解为面向行为的内部承诺准备状态读出，这挑战了将言语报告视为可靠性代理的做法。
英文摘要：
Confidence is an estimate of the probability that a chosen answer is correct. Verbal confidence reports are widely used as uncertainty measures in large language models, but whether they are best understood as estimates of correctness is unclear. We test this with a two-stage abstention paradigm from the neuroscience of perceptual decision making: a model first answers and reports its confidence, then decides whether to commit it to a user or abstain. Across four non-reasoning models, prompt framings, and confidence formats, verbal confidence predicted the commit/abstain decision substantially better than whether the answer was correct. Calibrated token log-probabilities showed the opposite profile, with abstention-prediction coupled to correctness discrimination, the signature of an answer-evidence signal. After removing the variance verbal confidence shared with log-probabilities, the residual stayed aligned with commitment while its link to correctness fell to near chance. The dissociation generalised to four reasoning models across four benchmarks of varying difficulty, from hard multiple-choice to frontier-level freeform questions. Mechanistic analyses in Gemma 3 and 4 were convergent: a post-answer state known to causally support verbal-confidence generation already encoded the future abstention decision before the abstention prompt, organised mainly by that decision rather than by correctness, the two lying in approximately orthogonal directions in activation space. Steering along a verbal-confidence-specific direction causally shifted abstention. Verbal and log-probability confidence are thus not interchangeable: log-probabilities track answer evidence and correctness, whereas verbal confidence is better understood as a behaviour-facing readout of an internal commit-readiness state, challenging the practice of treating verbal reports as proxies for reliability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
