大型语言模型中用于数学验证的前向-后向推理
Forward-Backward Reasoning in Large Language Models for Mathematical Verification
AI总结：
针对Self-Consistency前向推理饱和问题，提出FOBAR方法结合前向与后向推理验证候选答案，在多个数学数据集上达到SOTA，并验证了其有效性与扩展性。
摘要：
Self-Consistency（自洽性）采样具有答案的多样化推理链，并通过多数投票选择最终答案。它基于前向推理，当饱和时无法通过采样更多推理链来进一步提升性能。为了进一步提升性能，我们引入后向推理来验证候选答案。具体而言，对于数学任务，我们掩盖问题中的一个数字，并要求LLM回答由简单模板创建的后向问题，即在提供候选答案时预测被掩盖的数字。我们提出FOBAR，将前向和后向推理结合用于验证，而不是单独使用前向或后向推理。在六个标准数学数据集和三个LLM上的大量实验表明，FOBAR达到了最先进的性能。特别是，FOBAR优于仅使用前向推理的Self-Consistency，证明结合前向和后向推理效果更好。此外，FOBAR优于现有验证方法，表明后向推理中使用的简单模板及所提组合的有效性。对非数学问题的扩展也进行了讨论和实验验证。
英文摘要：
Self-Consistency samples diverse reasoning chains with answers and chooses the final answer by majority voting. It is based on forward reasoning and cannot further improve performance by sampling more reasoning chains when saturated. To further boost performance, we introduce backward reasoning to verify candidate answers. Specifically, for mathematical tasks, we mask a number in the question and ask the LLM to answer a backward question created by a simple template, i.e., to predict the masked number when a candidate answer is provided. Instead of using forward or backward reasoning alone, we propose FOBAR to combine FOrward and BAckward Reasoning for verification. Extensive experiments on six standard mathematical data sets and three LLMs show that FOBAR achieves state-of-the-art performance. In particular, FOBAR outperforms Self-Consistency, which uses forward reasoning alone, demonstrating that combining forward and forward reasoning is better. In addition, FOBAR performs better than existing verification methods, showing the effectiveness of the simple template used in backward reasoning and the proposed combination. Extensions to non-mathematical problems are also discussed and validated empirically.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
