学习检查：释放大语言模型自我纠正的潜力
Learning to Check: Unleashing Potentials for Self-Correction in Large Language Models
AI总结：
本文提出“Step CoT Check”检查格式并构建检查-纠正数据集，通过微调增强大语言模型在推理任务中的自我检查与自我纠正能力，在多个基准上显著优于其他格式，且在更大模型中收益更明显。
摘要：
自我纠正已在提升大语言模型（LLMs）生成输出的风格与安全性方面取得了显著成果。然而，近期研究表明，由于LLMs难以识别逻辑错误，自我纠正在推理任务中可能作用有限，甚至产生反效果。本文旨在通过构建用于检查任务的训练数据来增强LLMs的自我检查能力。具体而言，我们将思维链（Chain of Thought，CoT）方法应用于自我检查任务，利用细粒度的步骤级分析和解释来评估推理路径的正确性。我们提出了一种专门的检查格式，称为“Step CoT Check”。按照该格式，我们构建了一个包含详细逐步分析与检查的检查-纠正数据集。随后，我们对LLMs进行微调，以增强其错误检测和纠正能力。实验表明，采用“Step CoT Check”格式进行微调可显著提升LLMs在多个基准上的自我检查和自我纠正能力。该方法优于其他格式，尤其在定位错误位置方面表现突出，并且在更大的模型中观察到更显著的收益。为便于复现，所有数据集和代码已提供于 https://github.com/bammt/Learn-to-check。
英文摘要：
Self-correction has achieved impressive results in enhancing the style and security of the generated output from large language models (LLMs). However, recent studies suggest that self-correction might be limited or even counterproductive in reasoning tasks due to LLMs' difficulties in identifying logical mistakes. In this paper, we aim to enhance the self-checking capabilities of LLMs by constructing training data for checking tasks. Specifically, we apply the Chain of Thought (CoT) methodology to self-checking tasks, utilizing fine-grained step-level analyses and explanations to assess the correctness of reasoning paths. We propose a specialized checking format called "Step CoT Check". Following this format, we construct a checking-correction dataset that includes detailed step-by-step analysis and checking. Then we fine-tune LLMs to enhance their error detection and correction abilities. Our experiments demonstrate that fine-tuning with the "Step CoT Check" format significantly improves the self-checking and self-correction abilities of LLMs across multiple benchmarks. This approach outperforms other formats, especially in locating the incorrect position, with greater benefits observed in larger models. For reproducibility, all the datasets and code are provided in https://github.com/bammt/Learn-to-check.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
