LLM推理链中准确性的词汇提示
Lexical Hints of Accuracy in LLM Reasoning Chains
AI总结：
本文研究LLM思维链中的词汇提示能否预测答案正确性，发现不确定性词汇是错误回答的最强信号，而CoT长度仅在中等难度基准上有效，为轻量级事后校准提供依据。
摘要：
通过强化学习微调大语言模型（LLMs），使其在回答前生成显式的思维链（Chain-of-Thought, CoT），可以持续提升模型在代码、数学和通用知识基准上的整体表现。然而，在LLMs目前准确率较低的基准上，例如Humanity's Last Exam (HLE)，模型往往报告较高的自信度，反映出校准效果不佳。本文测试了CoT的可测量属性是否能作为LLM对其答案内部置信度的可靠信号。我们分析了三类特征：（i）CoT长度，（ii）CoT内部情感波动，以及（iii）词汇提示，包括模糊限定词。使用DeepSeek-R1和Claude 3.7 Sonnet在Humanity's Last Exam (HLE)（一个准确率极低的前沿基准）和Omni-MATH（一个中等难度的饱和基准）上进行实验，我们发现CoT中的不确定性词汇标记（例如$\textit{guess}$、$\textit{stuck}$、$\textit{hard}$）是错误回答的最强指示信号，而CoT情感的变化提供了较弱但互补的信号。CoT长度仅在Omni-MATH上具有信息量，该基准的准确率已经较高（$\approx 70\\%$），而在更难的HLE（$\approx 9\\%$）上不携带任何信号，这表明CoT长度仅在中等难度基准中预测正确性，即处于模型已展示的能力范围内但仍未达到饱和状态。最后，我们发现CoT中的不确定性指示信号始终比高置信度标记更为显著，使得错误比正确回答更容易预测。我们的研究结果支持一种轻量级的事后校准信号，该信号可以补充不可靠的自我报告概率，并支持更安全地部署LLMs。
英文摘要：
Fine-tuning Large Language Models (LLMs) with reinforcement learning to produce an explicit Chain-of-Thought (CoT) before answering produces models that consistently raise overall performance on code, math, and general-knowledge benchmarks. However, on benchmarks where LLMs currently achieve low accuracy, such as Humanity's Last Exam (HLE), they often report high self-confidence, reflecting poor calibration. Here, we test whether measurable properties of the CoT provide reliable signals of an LLM's internal confidence in its answers. We analyze three feature classes: (i) CoT length, (ii) intra-CoT sentiment volatility, and (iii) lexicographic hints, including hedging words. Using DeepSeek-R1 and Claude 3.7 Sonnet on both Humanity's Last Exam (HLE), a frontier benchmark with very low accuracy, and Omni-MATH, a saturated benchmark of moderate difficulty, we find that lexical markers of uncertainty (e.g., $\textit{guess}$, $\textit{stuck}$, $\textit{hard}$) in the CoT are the strongest indicators of an incorrect response, while shifts in the CoT sentiment provide a weaker but complementary signal. CoT length is informative only on Omni-MATH, where accuracy is already high ($\approx 70\%$), and carries no signal on the harder HLE ($\approx 9\%$), indicating that CoT length predicts correctness only in the intermediate-difficulty benchmarks, i.e., inside the model's demonstrated capability, but still below saturation. Finally, we find that uncertainty indicators in the CoT are consistently more salient than high-confidence markers, making errors easier to predict than correct responses. Our findings support a lightweight post-hoc calibration signal that complements unreliable self-reported probabilities and supports safer deployment of LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
