为自信所束缚：对大语言模型中过度自信的机械视角分析
Wired for Overconfidence: A Mechanistic Perspective on Inflated Verbalized Confidence in LLMs
AI总结：
研究探讨了大语言模型中过度自信的机制，发现中间层的MLP块和注意力头会生成自信信号，并通过干预提升校准性。
摘要：
大型语言模型往往不仅错误，而且自信地错误：当它们产生事实性错误答案时，倾向于表达过高的自信而非信号不确定性。这种表达的过度自信可能误导用户并削弱自信分数作为可靠不确定性信号的作用，但其内部机制仍不明确。本文对LLM中的过度自信进行了电路级机械分析，围绕三个轴展开：将表达自信作为可微内部信号、识别因果放大其的电路、以及利用这些见解进行针对性推理时校准。在两个指令调优的LLM上三个数据集上，发现一组紧凑的MLP块和注意力头，集中在中后期层，持续在最终标记位置写入自信膨胀信号。我们进一步表明，对这些电路的针对性推理时干预显著提高了校准性。总体而言，我们的结果表明，LLM中的表达过度自信由可识别的内部电路驱动，可通过针对性干预缓解。
英文摘要：
Large language models are often not just wrong, but \emph{confidently wrong}: when they produce factually incorrect answers, they tend to verbalize overly high confidence rather than signal uncertainty. Such verbalized overconfidence can mislead users and weaken confidence scores as a reliable uncertainty signal, yet its internal mechanisms remain poorly understood. We present a circuit-level mechanistic analysis of this inflated verbalized confidence in LLMs, organized around three axes: capturing verbalized confidence as a differentiable internal signal, identifying the circuits that causally inflate it, and leveraging these insights for targeted inference-time recalibration. Across two instruction-tuned LLMs on three datasets, we find that a compact set of MLP blocks and attention heads, concentrated in middle-to-late layers, consistently writes the confidence-inflation signal at the final token position. We further show that targeted inference-time interventions on these circuits substantially improve calibration. Together, our results suggest that verbalized overconfidence in LLMs is driven by identifiable internal circuits and can be mitigated through targeted intervention.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
