arXiv 2504.213182025-04-30cs.AIcs.CL

Phi-4-reasoning 技术报告

Phi-4-reasoning Technical Report

Microsoft（微软）

Marah Abdin, Sahaj Agarwal, Ahmed Awadallah, Vidhisha Balachandran, Harkirat Behl, Lingjiao Chen, Gustavo de Rosa, Suriya Gunasekar, Mojan Javaheripi, Neel Josh… 展开作者

Marah Abdin, Sahaj Agarwal, Ahmed Awadallah, Vidhisha Balachandran, Harkirat Behl, Lingjiao Chen, Gustavo de Rosa, Suriya Gunasekar, Mojan Javaheripi, Neel Joshi, Piero Kauffmann, Yash Lara, Caio César Teodoro Mendes, Arindam Mitra, Besmira Nushi, Dimitris Papailiopoulos, Olli Saarikivi, Shital Shah, Vaishnavi Shrivastava, Vibhav Vineet, Yue Wu, Safoora Yousefi, Guoqing Zheng

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文介绍了140亿参数的推理模型Phi-4-reasoning及其强化学习增强变体，通过精心筛选的数据进行监督微调，在多项推理任务上超越更大模型并接近DeepSeek-R1水平。

摘要：

我们介绍了 Phi-4-reasoning，这是一个拥有 140 亿参数的推理模型，在复杂推理任务上取得了强劲的表现。该模型通过对 Phi-4 进行监督微调（SFT）训练而成，使用了精心筛选的“可教学”提示集（因其适当的复杂性和多样性而被选中），以及使用 o3-mini 生成的推理演示。Phi-4-reasoning 能够生成详细的推理链，从而有效利用推理时计算资源。我们进一步开发了 Phi-4-reasoning-plus，这是一个通过短暂的基于结果的强化学习（RL）阶段增强的变体，它通过生成更长的推理轨迹来提供更高的性能。在广泛的推理任务中，这两个模型的表现均显著优于规模大得多的开放权重模型（如 DeepSeek-R1-Distill-Llama-70B 模型），并接近完整的 DeepSeek-R1 模型的性能水平。我们的综合评估涵盖了数学和科学推理、编程、算法问题解决、规划和空间理解等基准测试。有趣的是，我们观察到这些改进向通用基准测试也产生了非平凡的迁移。在本报告中，我们提供了关于训练数据、训练方法和评估的见解。我们表明，针对监督微调（SFT）进行仔细数据筛选的益处延伸到了推理语言模型上，并且可以通过强化学习（RL）进一步放大。最后，我们的评估指出了在改进我们如何评估推理模型的性能和鲁棒性方面存在的机遇。

英文摘要：

We introduce Phi-4-reasoning, a 14-billion parameter reasoning model that achieves strong performance on complex reasoning tasks. Trained via supervised fine-tuning of Phi-4 on carefully curated set of "teachable" prompts-selected for the right level of complexity and diversity-and reasoning demonstrations generated using o3-mini, Phi-4-reasoning generates detailed reasoning chains that effectively leverage inference-time compute. We further develop Phi-4-reasoning-plus, a variant enhanced through a short phase of outcome-based reinforcement learning that offers higher performance by generating longer reasoning traces. Across a wide range of reasoning tasks, both models outperform significantly larger open-weight models such as DeepSeek-R1-Distill-Llama-70B model and approach the performance levels of full DeepSeek-R1 model. Our comprehensive evaluations span benchmarks in math and scientific reasoning, coding, algorithmic problem solving, planning, and spatial understanding. Interestingly, we observe a non-trivial transfer of improvements to general-purpose benchmarks as well. In this report, we provide insights into our training data, our training methodologies, and our evaluations. We show that the benefit of careful data curation for supervised fine-tuning (SFT) extends to reasoning language models, and can be further amplified by reinforcement learning (RL). Finally, our evaluation points to opportunities for improving how we assess the performance and robustness of reasoning models.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。