arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.20770cs.AI
收藏

验证大语言模型生成数学解的管道

Pipeline for Verifying LLM-Generated Mathematical Solutions

Moscow State University(莫斯科国立大学);Huawei Technologies Co., Ltd(华为技术有限公司)

Varvara Sazonova, Dmitri Shmelkin, Stanislav Kikot, Vasily Motolygin

AI总结:

本研究提出了一种验证大语言模型数学解的管道,通过提示生成特定形式的解以提高验证准确性,并提供开源实现。

摘要:

随着大型推理模型日益流行及其在解决数学问题上的成果,测量其能力变得至关重要。我们介绍了一种用于自动和交互验证的管道,作为比仅检查答案这一目前最流行的方法更准确的替代方案。该管道还可以用作在正式和非正式语言中生成正确解的工具。结构中包含三个AI代理,可以根据基准需求进行选择。关键思想是使用提示以特定形式获取解,从而更易于使用证明助手进行验证,并可能使用小模型(≤8B)。在多个数据集上的实验表明假阳性概率较低。开源实现及服务器设置说明可在https://github.com/LogicEnj/lean4_verification_pipeline上找到。

英文摘要:

With the growing popularity of Large Reasoning Models and their results in solving mathematical problems, it becomes crucial to measure their capabilities. We introduce a pipeline for both automatic and interactive verification as a more accurate alternative to only checking the answer which is currently the most popular approach for benchmarks. The pipeline can also be used as a generator of correct solutions both in formal and informal languages. 3 AI agents, which can be chosen for the benchmark accordingly, are included in the structure. The key idea is the use of prompts to obtain the solution in the specific form which allows for easier verification using proof assistants and possible use of small models ($\le 8B$). Experiments on several datasets suggest low probability of False Positives. The open-source implementation with instructions on setting up a server is available at https://github.com/LogicEnj/lean4_verification_pipeline.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑