重新思考思维链：上下文学习与预训练先验的作用
Rethinking the Chain-of-Thought: The Roles of In-Context Learning and Pre-trained Priors
AI总结：
该研究从上下文学习与预训练先验的双重关系视角探究思维链推理机制，通过细粒度分析、噪声示例引入等实验得出三项关键发现，为理解思维链推理原理及优化提示工程提供了依据。
摘要：
思维链推理已成为提升模型推理能力的关键方法。尽管学界对思维链推理的兴趣日益浓厚，但其底层机制仍不明确。本文从上下文学习与预训练先验的双重关系视角出发，探究思维链推理的工作机制。我们首先对推理依据进行细粒度的词汇层面分析，以考察模型的推理行为。随后，通过逐步引入噪声示例，检验模型如何在预训练先验与错误的上下文信息之间取得平衡。最后，我们探究提示工程能否激发大语言模型的慢思考。大量实验得出三项关键发现：（1）模型不仅能在词汇层面快速学习推理结构，还能掌握更深层的逻辑推理模式，但严重依赖预训练先验。（2）提供充足示例可使模型的决策从依赖预训练先验转向依赖上下文信号，而误导性提示会引入不稳定性。（3）长思维链提示可促使模型生成更长的推理链，从而提升其在下游任务上的表现。
英文摘要：
Chain-of-Thought reasoning has emerged as a pivotal methodology for enhancing model inference capabilities. Despite growing interest in Chain-of-Thought reasoning, its underlying mechanisms remain unclear. This paper explores the working mechanisms of Chain-of-Thought reasoning from the perspective of the dual relationship between in-context learning and pretrained priors. We first conduct a fine-grained lexical-level analysis of rationales to examine the model's reasoning behavior. Then, by incrementally introducing noisy exemplars, we examine how the model balances pretrained priors against erroneous in-context information. Finally, we investigate whether prompt engineering can induce slow thinking in large language models. Our extensive experiments reveal three key findings: (1) The model not only quickly learns the reasoning structure at the lexical level but also grasps deeper logical reasoning patterns, yet it heavily relies on pretrained priors. (2) Providing sufficient exemplars shifts the model's decision-making from pretrained priors to in-context signals, while misleading prompts introduce instability. (3) Long Chain-of-Thought prompting can induce the model to generate longer reasoning chains, thereby improving its performance on downstream tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
