PromptCoT 2.0：面向大语言模型推理的提示合成规模化扩展
PromptCoT 2.0: Scaling Prompt Synthesis for Large Language Model Reasoning
AI总结：
PromptCoT 2.0 通过期望最大化循环迭代精炼推理链来合成更难、更多样化的训练提示，支持自博弈和监督微调，在多个基准上显著提升模型推理性能。
摘要：
大语言模型（LLMs）正从对话系统演变为强大的推理器，能够处理如奥林匹克数学和竞争性编程等任务。虽然扩展参数规模和测试时计算已推动进展，但一个关键瓶颈是缺乏高质量的训练问题：人工策划的数据集成本高昂且数量有限，而现有的合成语料库往往过于简单或狭窄。PromptCoT 1.0 表明，在提示合成中注入推理链（rationales）能增加问题难度。在此基础上，我们提出了 PromptCoT 2.0，一个可扩展的框架，用期望最大化（EM）循环取代手工设计的启发式方法，其中推理链被迭代精炼以指导提示构建。这产生的问题比之前的语料库更难且更多样化。合成提示支持两种后训练模式：（1）自博弈（Self-Play），其中强模型通过可验证反馈自主改进，无需更强的教师模型；（2）监督微调（SFT），其中较弱模型从教师蒸馏的轨迹中学习。大量实验证明了该方法的有效性。在自博弈中，将 PromptCoT 2.0 应用于 Qwen3-30B-A3B-Thinking-2507，在 30B 规模上取得了新的最先进结果，在 AIME 24/25 和 HMMT 25 上分别提升 +4.4、+4.8 和 +5.3，在 LiveCodeBench v5/v6 上提升 +6.1 和 +5.0，在 Codeforces 上提升 +35 Elo。在 SFT 中，仅使用合成提示训练 Qwen2.5-7B-Instruct，将准确率提升至 73.1（AIME 24）、65.6（AIME 25）和 53.4（LiveCodeBench v5），超过了在人工或混合数据上训练的模型。进一步分析证实，PromptCoT 2.0 产生的问题本质上更难且分布上更独特。这些结果确立了提示合成作为扩展推理的新维度，并将 PromptCoT 2.0 定位为未来开源模型的可扩展基础。实现可在 https://github.com/inclusionAI/PromptCoT 获取。
英文摘要：
Large language models (LLMs) are evolving from conversational systems into strong reasoners for tasks such as Olympiad mathematics and competitive programming. While scaling parameters and test-time computation has driven progress, a key bottleneck is the lack of high-quality training problems: human-curated datasets are costly and limited, while existing synthetic corpora are often too easy or narrow. PromptCoT 1.0 showed that injecting rationales into prompt synthesis increases problem difficulty. Building on this, we present PromptCoT 2.0, a scalable framework that replaces hand-crafted heuristics with an expectation-maximization (EM) loop, where rationales are iteratively refined to guide prompt construction. This produces problems that are both harder and more diverse than prior corpora. The synthetic prompts support two post-training regimes: (1) Self-Play, where strong models improve autonomously via verifiable feedback without stronger teachers; and (2) Supervised Fine-Tuning (SFT), where weaker models learn from teacher-distilled traces. Extensive experiments demonstrate the effectiveness of this approach. In self-play, applying PromptCoT 2.0 to Qwen3-30B-A3B-Thinking-2507 sets new state-of-the-art results at the 30B scale, with +4.4, +4.8, and +5.3 on AIME 24/25 and HMMT 25, +6.1 and +5.0 on LiveCodeBench v5/v6, and +35 Elo on Codeforces. In SFT, training Qwen2.5-7B-Instruct solely on synthetic prompts boosts accuracy to 73.1 (AIME 24), 65.6 (AIME 25), and 53.4 (LiveCodeBench v5), surpassing models trained on human or hybrid data. Analyses further confirm that PromptCoT 2.0 yields fundamentally harder and distributionally distinct problems. These results establish prompt synthesis as a new axis for scaling reasoning and position PromptCoT 2.0 as a scalable foundation for future open-source models. The implementation is available at https://github.com/inclusionAI/PromptCoT.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
