通过蒙特卡洛树自精炼与 LLaMa-3 8B 达到 GPT-4 级别数学奥林匹克解题能力
Accessing GPT-4 level Mathematical Olympiad Solutions via Monte Carlo Tree Self-refine with LLaMa-3 8B
AI总结:
本文提出 MCTSr 算法,将蒙特卡洛树搜索与自精炼机制结合到 LLaMa-3 8B 中,在多个数学基准上显著提升奥林匹克级解题成功率,达到 GPT-4 水平。
摘要:
本文提出了 MCT Self-Refine(MCTSr)算法,这是一种将大型语言模型(LLMs)与蒙特卡洛树搜索(MCTS)创新性结合的算法,旨在提升复杂数学推理任务的表现。针对 LLMs 在准确性和可靠性方面的挑战,尤其是在策略推理和数学推理中的挑战,MCTSr 利用系统性探索和启发式自精炼机制来改进 LLMs 内部的决策框架。该算法通过选择、自精炼、自评估和反向传播的迭代过程构建蒙特卡洛搜索树,并利用改进的上置信界(UCB)公式来优化探索与利用之间的平衡。大量实验证明了 MCTSr 在求解奥林匹克级别数学问题上的有效性,在多个数据集上显著提高了成功率,包括 GSM8K、GSM Hard、MATH 以及奥林匹克级别基准,包括 Math Odyssey、AIME 和 OlympiadBench。本研究推进了 LLMs 在复杂推理任务中的应用,并为未来 AI 集成奠定了基础,提升了 LLM 驱动应用中决策的准确性和可靠性。
英文摘要:
This paper introduces the MCT Self-Refine (MCTSr) algorithm, an innovative integration of Large Language Models (LLMs) with Monte Carlo Tree Search (MCTS), designed to enhance performance in complex mathematical reasoning tasks. Addressing the challenges of accuracy and reliability in LLMs, particularly in strategic and mathematical reasoning, MCTSr leverages systematic exploration and heuristic self-refine mechanisms to improve decision-making frameworks within LLMs. The algorithm constructs a Monte Carlo search tree through iterative processes of Selection, self-refine, self-evaluation, and Backpropagation, utilizing an improved Upper Confidence Bound (UCB) formula to optimize the exploration-exploitation balance. Extensive experiments demonstrate MCTSr's efficacy in solving Olympiad-level mathematical problems, significantly improving success rates across multiple datasets, including GSM8K, GSM Hard, MATH, and Olympiad-level benchmarks, including Math Odyssey, AIME, and OlympiadBench. The study advances the application of LLMs in complex reasoning tasks and sets a foundation for future AI integration, enhancing decision-making accuracy and reliability in LLM-driven applications.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。