思维链不忠实性：伪装的准确性
Chain-of-Thought Unfaithfulness as Disguised Accuracy
AI总结：
本研究复现了CoT忠实性缩放实验，发现归一化消除答案偏差后，小模型不忠实性显著下降，且归一化指标与准确率强相关，质疑了该指标评估忠实性的有效性。
摘要：
理解思维链（Chain-of-Thought, CoT）生成内容与大语言模型（large language model, LLM）内部计算的对齐程度，对判断是否可信LLM的输出至关重要。作为CoT忠实性的代理指标，Lanham等人（2023）提出了一种衡量模型生成答案时对其CoT依赖程度的指标。在单一专有模型家族中，他们发现LLM的模型规模与该忠实性度量值之间呈现“先缩放后逆缩放”的关系，且与参数规模在8.1亿到1750亿之间的模型相比，130亿参数模型的忠实性更高。我们评估了这些结果是否能推广为所有LLM的共性属性。我们使用三个不同的模型家族复现了其论文中聚焦缩放实验的章节的实验设置，并在特定条件下成功重现了他们报告的CoT忠实性缩放趋势。然而，在对该指标进行归一化以消除模型对特定答案选项的偏差后，规模较小、能力较弱的模型的不忠实性显著下降。这一归一化后的忠实性指标还与准确率强相关（$R^2$=0.74），引发了对其评估忠实性有效性的质疑。
英文摘要：
Understanding the extent to which Chain-of-Thought (CoT) generations align with a large language model's (LLM) internal computations is critical for deciding whether to trust an LLM's output. As a proxy for CoT faithfulness, Lanham et al. (2023) propose a metric that measures a model's dependence on its CoT for producing an answer. Within a single family of proprietary models, they find that LLMs exhibit a scaling-then-inverse-scaling relationship between model size and their measure of faithfulness, and that a 13 billion parameter model exhibits increased faithfulness compared to models ranging from 810 million to 175 billion parameters in size. We evaluate whether these results generalize as a property of all LLMs. We replicate the experimental setup in their section focused on scaling experiments with three different families of models and, under specific conditions, successfully reproduce the scaling trends for CoT faithfulness they report. However, after normalizing the metric to account for a model's bias toward certain answer choices, unfaithfulness drops significantly for smaller less-capable models. This normalized faithfulness metric is also strongly correlated ($R^2$=0.74) with accuracy, raising doubts about its validity for evaluating faithfulness.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
