PromptCoT: 为大型语言模型中的数学推理合成竞赛级问题
PromptCoT: Synthesizing Olympiad-level Problems for Mathematical Reasoning in Large Language Models
AI总结：
PromptCoT通过模拟竞赛级问题设计者的思维过程，自动生成高质量数学问题，提升大型语言模型的数学推理能力。
摘要：
大型语言模型解决复杂数学问题的能力已显著提升，特别是在需要高级推理的任务中。然而，足够具有挑战性的问题稀缺，尤其是竞赛级别的问题，这阻碍了进一步的进步。在本文中，我们介绍了PromptCoT，一种新的方法，用于自动生成高质量的竞赛级数学问题。所提出的方法基于数学概念和问题构造的原理来合成复杂问题，模拟有经验的问题设计者的思维过程。我们提供了一个理论分析，证明最优的原理应同时最大化在给定相关概念下的原理生成概率以及在原理和概念条件下问题生成的概率。我们的方法在标准基准上进行了评估，包括GSM8K、MATH-500和AIME2024，在这些基准上它始终优于现有问题生成方法。此外，我们证明PromptCoT在数据可扩展性方面表现优异，随着数据集规模的增加，其性能保持稳定，优于基线方法。实现代码可在https://github.com/zhaoxlpku/PromptCoT上找到。
英文摘要：
The ability of large language models to solve complex mathematical problems has progressed significantly, particularly for tasks requiring advanced reasoning. However, the scarcity of sufficiently challenging problems, particularly at the Olympiad level, hinders further advancements. In this work, we introduce PromptCoT, a novel approach for automatically generating high-quality Olympiad-level math problems. The proposed method synthesizes complex problems based on mathematical concepts and the rationale behind problem construction, emulating the thought processes of experienced problem designers. We provide a theoretical analysis demonstrating that an optimal rationale should maximize both the likelihood of rationale generation given the associated concepts and the likelihood of problem generation conditioned on both the rationale and the concepts. Our method is evaluated on standard benchmarks including GSM8K, MATH-500, and AIME2024, where it consistently outperforms existing problem generation methods. Furthermore, we demonstrate that PromptCoT exhibits superior data scalability, consistently maintaining high performance as the dataset size increases, outperforming the baselines. The implementation is available at https://github.com/zhaoxlpku/PromptCoT.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
