FACT-E：基于因果的可信推理链评估
FACT-E: Causality-Inspired Evaluation for Trustworthy Chain-of-Thought Reasoning
AI总结：
FACT-E通过因果方法评估推理链的可信度，结合内部一致性与推理-答案一致性，提升推理轨迹选择质量并增强噪声下的鲁棒性。
摘要：
链式思维（CoT）提示方法提升了大语言模型的推理能力，但模型生成的解释往往看似连贯却包含不忠实的中间步骤。现有的自我评估方法容易受到固有偏见的影响：模型可能自信地认可连贯性，即使步骤间的推导不成立，导致不忠实性评估不可靠。我们提出了FACT-E，一种基于因果的框架，用于评估CoT质量。FACT-E利用受控扰动作为仪器信号，将真实的步骤间依赖与偏见驱动的伪影区分开来，产生更可靠的忠实性估计（链内忠实性）。为选择可信的轨迹，FACT-E同时考虑链内忠实性和CoT到答案的一致性，确保所选链在内部忠实且支持正确的最终答案。在GSM8K、MATH和常识问答数据集上的实验表明，FACT-E提高了推理轨迹选择，并产生了更强的上下文学习示例。FACT-E还能在噪声条件下可靠地检测到错误推理，提供了一个稳健的指标，用于可信的大语言模型推理。
英文摘要：
Chain-of-Thought (CoT) prompting has improved LLM reasoning, but models often generate explanations that appear coherent while containing unfaithful intermediate steps. Existing self-evaluation approaches are prone to inherent biases: the model may confidently endorse coherence even when the step-to-step implication is not valid, leading to unreliable faithfulness evaluation. We propose FACT-E, a causality-inspired framework for evaluating CoT quality. FACT-E uses controlled perturbations as an instrumental signal to separate genuine step-to-step dependence from bias-driven artifacts, producing more reliable faithfulness estimates (\textit{intra-chain faithfulness}). To select trustworthy trajectories, FACT-E jointly considers \textit{intra-chain faithfulness} and \textit{CoT-to-answer consistency}, ensuring that selected chains are both faithful internally and supportive of the correct final answer. Experiments on GSM8K, MATH, and CommonsenseQA show that FACT-E improves reasoning-trajectory selection and yields stronger in-context learning exemplars. FACT-E also reliably detects flawed reasoning under noisy conditions, providing a robust metric for trustworthy LLM reasoning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
