弥合大语言模型中的置信度-忠实度差距
Closing the Confidence-Faithfulness Gap in Large Language Models
AI总结：
本文通过线性探针和对比激活添加分析，揭示大语言模型中置信度信号与校准的线性关系，提出双阶段适应性引导流程以提升校准对齐。
摘要：
大语言模型（LLMs）倾向于生成与实际准确性脱节的置信度分数，但支配这一行为的几何关系尚不明确。本文通过线性探针和对比激活添加（CAA）引导分析，发现校准和生成置信度信号以线性方式编码但彼此正交，这一发现在三个开源模型和四个数据集中均一致。有趣的是，当模型被提示同时进行推理并生成置信度分数时，推理过程会破坏生成置信度方向，加剧校准偏差。我们称之为'推理污染效应'。利用这一见解，我们引入了双阶段适应性引导流程，读取模型内部的准确性估计并引导生成输出以匹配，从而在所有评估模型中显著提升校准对齐。
英文摘要：
Large language models (LLMs) tend to verbalize confidence scores that are largely detached from their actual accuracy, yet the geometric relationship governing this behavior remain poorly understood. In this work, we present a mechanistic interpretability analysis of verbalized confidence, using linear probes and contrastive activation addition (CAA) steering to show that calibration and verbalized confidence signals are encoded linearly but are orthogonal to one another -- a finding consistent across three open-weight models and four datasets. Interestingly, when models are prompted to simultaneously reason through a problem and verbalize a confidence score, the reasoning process disrupts the verbalized confidence direction, exacerbating miscalibration. We term this the "Reasoning Contamination Effect." Leveraging this insight, we introduce a two-stage adaptive steering pipeline that reads the model's internal accuracy estimate and steers verbalized output to match it, substantially improving calibration alignment across all evaluated models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
