arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.06195cs.CL
收藏

互推理让更小的大语言模型成为更强的问题解决者

Mutual Reasoning Makes Smaller LLMs Stronger Problem-Solvers

Microsoft Research Asia(微软亚洲研究院);Harvard University(哈佛大学)

Zhenting Qi, Mingyuan Ma, Jiahang Xu, Li Lyna Zhang, Fan Yang, Mao Yang

AI总结:

本研究提出无需微调的自博弈互推理方法rStar,通过生成-判别过程增强小语言模型推理能力,在多个基准上实现显著准确率提升。

摘要:

本文提出了rStar,一种自博弈互推理方法,无需微调或依赖更优模型即可显著提升小语言模型(SLMs)的推理能力。rStar将推理解耦为自博弈的互生成-判别过程。首先,目标SLM用丰富的类人推理动作增强蒙特卡洛树搜索(MCTS),构建更高质量的推理轨迹。随后,另一台能力与目标SLM相近的SLM作为判别器,验证目标SLM生成的每条轨迹。双方达成一致的推理轨迹被认为具有互一致性,因此更有可能正确。针对五款SLM的大量实验表明,rStar可有效解决各类推理问题,涵盖GSM8K、GSM-Hard、MATH、SVAMP和StrategyQA。值得注意的是,rStar将LLaMA2-7B的GSM8K准确率从12.51%提升至63.91%,Mistral-7B从36.46%提升至81.88%,LLaMA3-8B-Instruct从74.53%提升至91.13%。代码将发布于https://github.com/zhentingqi/rStar。

英文摘要:

This paper introduces rStar, a self-play mutual reasoning approach that significantly improves reasoning capabilities of small language models (SLMs) without fine-tuning or superior models. rStar decouples reasoning into a self-play mutual generation-discrimination process. First, a target SLM augments the Monte Carlo Tree Search (MCTS) with a rich set of human-like reasoning actions to construct higher quality reasoning trajectories. Next, another SLM, with capabilities similar to the target SLM, acts as a discriminator to verify each trajectory generated by the target SLM. The mutually agreed reasoning trajectories are considered mutual consistent, thus are more likely to be correct. Extensive experiments across five SLMs demonstrate rStar can effectively solve diverse reasoning problems, including GSM8K, GSM-Hard, MATH, SVAMP, and StrategyQA. Remarkably, rStar boosts GSM8K accuracy from 12.51% to 63.91% for LLaMA2-7B, from 36.46% to 81.88% for Mistral-7B, from 74.53% to 91.13% for LLaMA3-8B-Instruct. Code will be available at https://github.com/zhentingqi/rStar.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑