理解思维链提示：什么才是关键的实证研究
Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters
AI总结：
本文通过实证研究CoT提示的有效因素，发现无效推理示范仍可保留其80%至90%以上性能，而相关性和步骤顺序更关键。
摘要：
Chain-of-Thought（CoT，思维链）提示能够显著提升大型语言模型（LLM）的多步推理能力。CoT通过在示范中提供一系列推理步骤，明确鼓励LLM生成用于解决问题的中间理由。尽管CoT取得了成功，人们仍然不太理解是什么使CoT提示有效，以及所示范推理步骤的哪些方面促成了其性能。本文表明，即使使用无效示范，CoT推理也是可能的：在多种指标下，使用无效推理步骤进行提示可达到有效CoT所获性能的80%至90%以上，并且在推理过程中仍能生成连贯的推理脉络。进一步实验表明，理由的其他方面——例如与查询相关以及推理步骤顺序正确——对于有效的CoT推理要重要得多。总体而言，这些发现既加深了我们对CoT提示的理解，也提出了关于LLM在上下文中学习推理能力的新问题。
英文摘要：
Chain-of-Thought (CoT) prompting can dramatically improve the multi-step reasoning abilities of large language models (LLMs). CoT explicitly encourages the LLM to generate intermediate rationales for solving a problem, by providing a series of reasoning steps in the demonstrations. Despite its success, there is still little understanding of what makes CoT prompting effective and which aspects of the demonstrated reasoning steps contribute to its performance. In this paper, we show that CoT reasoning is possible even with invalid demonstrations - prompting with invalid reasoning steps can achieve over 80-90% of the performance obtained using CoT under various metrics, while still generating coherent lines of reasoning during inference. Further experiments show that other aspects of the rationales, such as being relevant to the query and correctly ordering the reasoning steps, are much more important for effective CoT reasoning. Overall, these findings both deepen our understanding of CoT prompting, and open up new questions regarding LLMs' capability to learn to reason in context.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
