Critic-CoT：通过思维链批评提升大语言模型的推理能力
Critic-CoT: Boosting the reasoning abilities of large language model via Chain-of-thoughts Critic
AI总结：
针对现有LLM自我批评方法偏直觉、推理受限的问题，提出Critic-CoT框架，通过逐步CoT推理和自动构建无标注远程监督数据实现类系统2的深度自我修正，在GSM8K、MATH上显著提升性能，并证实两类能力可相互强化。
摘要：
自我批评已成为增强大语言模型（LLM）推理性能的关键机制。然而，当前方法主要依赖基础提示生成直观的实例级反馈，这类似于系统1（System-1）的处理过程，限制了推理能力的提升。此外，现有研究缺乏对LLM批评能力与其任务求解性能之间关系的深入探究。为解决这些问题，我们提出Critic-CoT——一种推动LLM具备类系统2（System-2）批评能力的新型框架。通过逐步的思维链（CoT）推理范式，以及无需人工标注的远程监督数据自动构建方法，Critic-CoT使LLM能够进行缓慢、分析性的自我批评与修正，从而提升其推理能力。在GSM8K和MATH数据集上的实验表明，我们的增强模型通过过滤无效解或迭代修正，显著提升了任务求解性能。此外，我们探究了LLM内部批评能力与任务求解能力的内在关联，发现二者可相互强化而非相互冲突。
英文摘要：
Self-critic has become a crucial mechanism for enhancing the reasoning performance of LLMs. However, current approaches mainly involve basic prompts for intuitive instance-level feedback, which resembles System-1 processes and limits the reasoning capabilities. Moreover, there is a lack of in-depth investigations into the relationship between LLM's ability to criticize and its task-solving performance. To address these issues, we propose Critic-CoT, a novel framework that pushes LLMs toward System-2-like critic capability. Through a step-wise CoT reasoning paradigm and the automatic construction of distant-supervision data without human annotation, Critic-CoT enables LLMs to engage in slow, analytic self-critique and refinement, thereby improving their reasoning abilities. Experiments on GSM8K and MATH demonstrate that our enhanced model significantly boosts task-solving performance by filtering out invalid solutions or iterative refinement. Furthermore, we investigate the intrinsic correlation between critique and task-solving abilities within LLMs, discovering that these abilities can mutually reinforce each other rather than conflict.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
