迈向更好的思维链：对有效性与忠实性的反思
Towards Better Chain-of-Thought: A Reflection on Effectiveness and Faithfulness
AI总结：
本文从有效性和忠实性的角度研究了思维链的性能，识别了影响性能的关键因素，解释了不忠实CoT的问题，并提出了一种新算法，通过从问题中回忆额外信息来增强CoT生成并评估其信息增益，有效提升了CoT的性能。
摘要：
思维链提示在推理任务中表现出不同的性能。以往的工作试图对其进行评估，但在提供对影响 CoT 的提示的深入分析方面存在不足。在本文中，我们从有效性和忠实性的角度研究 CoT 的性能。对于前者，我们识别了影响 CoT 有效性以提升性能的关键因素，包括问题难度、信息增益和信息流。对于后者，我们通过对问题、CoT 和答案之间的信息交互进行联合分析，解释了不忠实的 CoT 问题。结果表明，当 LLM 预测答案时，它能够从问题中回忆起 CoT 中缺失的正确信息，从而导致该问题。最后，我们提出了一种新算法来缓解此问题，在该算法中，我们从问题中回忆额外信息以增强 CoT 生成，并基于此信息增益评估 CoT。大量实验证明，我们的方法在有效性和忠实性方面均提升了 CoT 的性能。
英文摘要：
Chain-of-thought (CoT) prompting demonstrates varying performance under different reasoning tasks. Previous work attempts to evaluate it but falls short in providing an in-depth analysis of patterns that influence the CoT. In this paper, we study the CoT performance from the perspective of effectiveness and faithfulness. For the former, we identify key factors that influence CoT effectiveness on performance improvement, including problem difficulty, information gain, and information flow. For the latter, we interpret the unfaithful CoT issue by conducting a joint analysis of the information interaction among the question, CoT, and answer. The result demonstrates that, when the LLM predicts answers, it can recall correct information missing in the CoT from the question, leading to the problem. Finally, we propose a novel algorithm to mitigate this issue, in which we recall extra information from the question to enhance the CoT generation and evaluate CoTs based on their information gain. Extensive experiments demonstrate that our approach enhances both the faithfulness and effectiveness of CoT.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
