RPM-MCTS：基于蒙特卡洛树搜索的知识检索作为过程奖励模型用于代码生成
RPM-MCTS: Knowledge-Retrieval as Process Reward Model with Monte Carlo Tree Search for Code Generation
AI总结：
RPM-MCTS通过结合知识检索与蒙特卡洛树搜索，有效评估代码生成过程中的中间步骤，减少错误并提升效率
摘要：
基于树搜索的方法在增强大语言模型的代码生成能力方面取得了显著进展。然而，由于难以有效评估中间算法步骤以及无法定位并及时纠正错误步骤，这些方法往往生成错误的代码并导致计算成本增加。为解决这些问题，我们提出了RPM-MCTS，一种基于蒙特卡洛树搜索的知识检索作为过程奖励模型的有效方法，用于评估中间算法步骤。通过利用知识库检索，RPM-MCTS避免了复杂的过程奖励模型训练。在扩展阶段，采用相似性过滤来移除冗余节点，确保推理路径的多样性。此外，我们的方法利用沙盒执行反馈在生成过程中定位错误的算法步骤，实现及时且有针对性的纠正。在四个公开的代码生成基准上的广泛实验表明，RPM-MCTS在超越当前最先进的方法的同时，实现了大约15%的标记消耗减少。此外，使用RPM-MCTS构建的数据对基础模型进行全微调显著增强了其代码能力。
英文摘要：
Tree search-based methods have made significant progress in enhancing the code generation capabilities of large language models. However, due to the difficulty in effectively evaluating intermediate algorithmic steps and the inability to locate and timely correct erroneous steps, these methods often generate incorrect code and incur increased computational costs. To tackle these problems, we propose RPM-MCTS, an effective method that utilizes Knowledge-Retrieval as Process Reward Model based on Monte Carlo Tree Search to evaluate intermediate algorithmic steps. By utilizing knowledge base retrieval, RPM-MCTS avoids the complex training of process reward models. During the expansion phase, similarity filtering is employed to remove redundant nodes, ensuring diversity in reasoning paths. Furthermore, our method utilizes sandbox execution feedback to locate erroneous algorithmic steps during generation, enabling timely and targeted corrections. Extensive experiments on four public code generation benchmarks demonstrate that RPM-MCTS outperforms current state-of-the-art methods while achieving an approximately 15% reduction in token consumption. Furthermore, full fine-tuning of the base model using the data constructed by RPM-MCTS significantly enhances its code capabilities.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
