论大语言模型中忠实思维链推理的困难性
On the Hardness of Faithful Chain-of-Thought Reasoning in Large Language Models
AI总结：
本文通过提出上下文学习、微调和激活编辑三类新策略并开展多基准实证分析，揭示现有方法在提升大语言模型思维链推理忠实性方面效果有限，表明该问题具有固有困难性。
摘要：
随着大语言模型（LLMs）日益被部署到医疗保健等关键领域的实际应用中，确保这些模型生成的思维链（Chain-of-Thought，CoT）推理忠实地反映其底层行为至关重要。尽管已知LLMs能够生成对人类具有吸引力的CoT推理，但已有研究表明，这些解释并不能准确反映底层LLMs的实际行为。在本工作中，我们探索了三种常用于引导LLM行为以增强其CoT推理忠实性的广泛方法的潜力：上下文学习（in-context learning）、微调（fine-tuning）和激活编辑（activation editing）。具体而言，我们提出了旨在提高CoT推理忠实性的新型上下文学习、微调和激活编辑策略。随后，我们在多个基准数据集上进行了广泛的实证分析，以探索这些策略的潜力。我们的分析表明，这些策略在提高CoT推理忠实性方面取得的成功有限，仅在受控场景中表现出轻微的性能提升。激活编辑取得的成功极小，而微调和上下文学习仅实现了边际改进，且未能泛化到不同的推理和真实问答基准上。总之，我们的工作凸显了从LLMs中引出忠实CoT推理的固有困难，表明当前的一系列方法可能不足以应对这一复杂挑战。
英文摘要：
As Large Language Models (LLMs) are increasingly being employed in real-world applications in critical domains such as healthcare, it is important to ensure that the Chain-of-Thought (CoT) reasoning generated by these models faithfully captures their underlying behavior. While LLMs are known to generate CoT reasoning that is appealing to humans, prior studies have shown that these explanations do not accurately reflect the actual behavior of the underlying LLMs. In this work, we explore the promise of three broad approaches commonly employed to steer the behavior of LLMs to enhance the faithfulness of the CoT reasoning generated by LLMs: in-context learning, fine-tuning, and activation editing. Specifically, we introduce novel strategies for in-context learning, fine-tuning, and activation editing aimed at improving the faithfulness of the CoT reasoning. We then carry out extensive empirical analyses with multiple benchmark datasets to explore the promise of these strategies. Our analyses indicate that these strategies offer limited success in improving the faithfulness of the CoT reasoning, with only slight performance enhancements in controlled scenarios. Activation editing demonstrated minimal success, while fine-tuning and in-context learning achieved marginal improvements that failed to generalize across diverse reasoning and truthful question-answering benchmarks. In summary, our work underscores the inherent difficulty in eliciting faithful CoT reasoning from LLMs, suggesting that the current array of approaches may not be sufficient to address this complex challenge.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
