置信度与批判力：大语言模型自我校正能力的分解
Confidence v.s. Critique: A Decomposition of Self-Correction Capability for LLMs
AI总结：
本文将大语言模型自我校正能力分解为置信度与批判力，提出概率化评估指标，通过实验揭示两者权衡关系，并提出改进SFT数据格式的策略以同时提升两种能力及校正后准确率。
摘要：
大语言模型（LLMs）能够校正其自身生成的回答，但同时也观察到自我校正后准确率下降的现象。为了更深入地理解自我校正，我们致力于分解、评估和分析大语言模型的自我校正行为。通过枚举并分析自我校正前后的答案正确性，我们将自我校正能力分解为置信度（对正确答案保持自信的能力）和批判力（将错误答案转变为正确答案的能力）两种能力，并从概率视角提出两个指标来衡量这两种能力，同时提出另一个指标用于整体自我校正能力评估。基于我们的分解方法和评估指标，我们进行了大量实验并得出一些经验性结论。例如，我们发现不同模型会表现出截然不同的行为：一些模型更自信，而另一些模型更具批判性。我们还发现，当通过提示词或上下文学习来操纵模型的自我校正行为时，这两种能力之间存在权衡关系（即提升一种能力可能导致另一种能力下降）。此外，我们发现了一种简单而高效的策略，通过转换监督微调（SFT）数据格式来提升自我校正能力，我们的策略在两种能力上均优于普通SFT，并在自我校正后取得了更高的准确率。我们的代码将在GitHub上公开。
英文摘要：
Large Language Models (LLMs) can correct their self-generated responses, but a decline in accuracy after self-correction is also witnessed. To have a deeper understanding of self-correction, we endeavor to decompose, evaluate, and analyze the self-correction behaviors of LLMs. By enumerating and analyzing answer correctness before and after self-correction, we decompose the self-correction capability into confidence (being confident to correct answers) and critique (turning wrong answers to correct) capabilities, and propose two metrics from a probabilistic perspective to measure these 2 capabilities, along with another metric for overall self-correction capability evaluation. Based on our decomposition and evaluation metrics, we conduct extensive experiments and draw some empirical conclusions. For example, we find different models can exhibit distinct behaviors: some models are confident while others are more critical. We also find the trade-off between the two capabilities (i.e. improving one can lead to a decline in the other) when manipulating model self-correction behavior by prompts or in-context learning. Further, we find a simple yet efficient strategy to improve self-correction capability by transforming Supervision Fine-Tuning (SFT) data format, and our strategy outperforms vanilla SFT in both capabilities and achieves much higher accuracy after self-correction. Our code will be publicly available on GitHub.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
