CoT的诅咒：论思维链在上下文学习中的局限性
The Curse of CoT: On the Limitations of Chain-of-Thought in In-Context Learning
AI总结：
本研究发现在基于模式的上下文学习中，CoT及其变体表现逊于直接回答，揭示其受显式-隐式推理混合机制制约，挑战了CoT普适有效性的假设。
摘要：
思维链（Chain-of-Thought, CoT）提示因增强大型语言模型（LLMs）推理能力而广受认可。然而，我们的研究在基于模式的上下文学习（ICL）这一基础领域内，揭示了一个与主流观点相悖的惊人矛盾。通过涉及16个最先进LLMs和9个多样化基于模式的ICL数据集的广泛实验，我们证明CoT及其推理变体在不同模型规模和基准复杂度下，表现始终逊于直接回答。为系统调查这一意外现象，我们设计了广泛实验以验证多种假设性解释。分析揭示了一种驱动CoT在基于模式的ICL中表现的显式-隐式推理混合机制：尽管显式推理因LLMs难以从示例中推断潜在模式而失效，但被CoT理由增加的上下文距离所干扰的隐式推理，往往能进行补偿，尽管理由存在缺陷仍能给出正确答案。这种混合机制解释了CoT的相对不佳表现，因为来自弱显式推断的噪声破坏了该过程，即便隐式机制部分挽救了结果。值得注意的是，即使在抽象和符号推理方面表现出色的长CoT推理模型，尽管计算成本更高，也无法完全克服这些限制。我们的发现挑战了关于CoT普适有效性的现有假设，为其局限性提供了新见解，并指引未来研究走向更细致有效的LLMs推理方法。
英文摘要：
Chain-of-Thought (CoT) prompting has been widely recognized for its ability to enhance reasoning capabilities in large language models (LLMs). However, our study reveals a surprising contradiction to this prevailing perspective within the fundamental domain of pattern-based in-context learning (ICL). Through extensive experiments involving 16 state-of-the-art LLMs and nine diverse pattern-based ICL datasets, we demonstrate that CoT and its reasoning variants consistently underperform direct answering across varying model scales and benchmark complexities. To systematically investigate this unexpected phenomenon, we designed extensive experiments to validate several hypothetical explanations. Our analysis uncovers a fundamental hybrid mechanism of explicit-implicit reasoning driving CoT's performance in pattern-based ICL: while explicit reasoning falters due to LLMs' struggles to infer underlying patterns from demonstrations, implicit reasoning-disrupted by the increased contextual distance of CoT rationales-often compensates, delivering correct answers despite flawed rationales. This hybrid mechanism explains CoT's relative underperformance, as noise from weak explicit inference undermines the process, even as implicit mechanisms partially salvage outcomes. Notably, even long-CoT reasoning models, which excel in abstract and symbolic reasoning, fail to fully overcome these limitations despite higher computational costs. Our findings challenge existing assumptions regarding the universal efficacy of CoT, yielding novel insights into its limitations and guiding future research toward more nuanced and effective reasoning methodologies for LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
