迈向基于领域知识的逐步推理优化与反思改进
Towards Stepwise Domain Knowledge-Driven Reasoning Optimization and Reflection Improvement
AI总结：
针对普通MCTS在特定领域专业知识任务中的挑战，提出基于领域知识的逐步推理优化框架及面向反思路径的偏好优化，实验证明其在法律领域的有效性并提供相关发现以推动领域研究。
摘要：
近期，借助蒙特卡洛树搜索（MCTS），对思维链（CoTs）的逐步监督在编码和数学等逻辑推理任务上展现出增强效果。然而，其在需要特定领域专业知识的任务中的作用尚未被探索。基于此兴趣，我们识别了普通MCTS在该语境下的若干潜在挑战，并提出基于领域知识的逐步推理优化框架，采用MCTS算法为需要必要理解、推理和专业知识的问题开发步骤级监督。此外，我们还引入面向反思路径的偏好优化，该方法从更好的视角迭代学习对推理思路的自我反思。我们开展了大量实验以评估这些方法的优势，实证结果证明其在各类法律领域问题上的有效性。我们还报告了一系列有价值的发现，希望能激发对特定领域大语言模型（LLMs）和MCTS研究的热情。
英文摘要：
Recently, stepwise supervision on Chain of Thoughts (CoTs) presents an enhancement on the logical reasoning tasks such as coding and math, with the help of Monte Carlo Tree Search (MCTS). However, its contribution to tasks requiring domain-specific expertise and knowledge remains unexplored. Motivated by the interest, we identify several potential challenges of vanilla MCTS within this context, and propose the framework of Stepwise Domain Knowledge-Driven Reasoning Optimization, employing the MCTS algorithm to develop step-level supervision for problems that require essential comprehension, reasoning, and specialized knowledge. Additionally, we also introduce the Preference Optimization towards Reflection Paths, which iteratively learns self-reflection on the reasoning thoughts from better perspectives. We have conducted extensive experiments to evaluate the advantage of the methodologies. Empirical results demonstrate the effectiveness on various legal-domain problems. We also report a diverse set of valuable findings, hoping to encourage the enthusiasm to the research of domain-specific LLMs and MCTS.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
