arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2502.06807cs.LGcs.AIcs.CL
收藏

使用大型推理模型进行竞技编程

Competitive Programming with Large Reasoning Models

OpenAI
OpenAI, :, Ahmed El-Kishky, Alexander Wei, Andre Saraiva, Borys Minaiev, Daniel Selsam, David Dohan, Francis Song, Hunter Lightman, Ignasi Clavera, Jakub Pachoc… 展开作者

OpenAI, :, Ahmed El-Kishky, Alexander Wei, Andre Saraiva, Borys Minaiev, Daniel Selsam, David Dohan, Francis Song, Hunter Lightman, Ignasi Clavera, Jakub Pachocki, Jerry Tworek, Lorenz Kuhn, Lukasz Kaiser, Mark Chen, Max Schwarzer, Mostafa Rohaninejad, Nat McAleese, o3 contributors, Oleg Mürk, Rhythm Garg, Rui Shu, Szymon Sidor, Vineet Kosaraju, Wenda Zhou

AI总结:

本文对比竞技编程专用系统 o1-ioi 与通用推理模型,证明扩展强化学习得到的 o3 无需手工策略即可在 IOI 获金牌并达到精英级 Codeforces 水平。

摘要:

我们表明,应用于大型语言模型(LLM)的强化学习可显著提升其在复杂编码和推理任务上的性能。此外,我们将两款通用推理模型——OpenAI o1 与 o3 的一个早期检查点——与一个领域专用系统 o1-ioi 进行了比较;o1-ioi 使用为参加 2024 年国际信息学奥林匹克(IOI)而设计的手工工程化推理策略。我们携 o1-ioi 现场参加了 IOI 2024,并借助手工制定的测试时策略,位列第 49 百分位。在放宽竞赛约束的条件下,o1-ioi 取得了金牌。然而,在评估 o3 等后续模型时,我们发现 o3 无需手工制定的领域专用策略或放宽约束即可获得金牌。我们的研究结果表明,尽管 o1-ioi 等专用流水线能带来切实提升,但规模扩大后的通用 o3 模型无需依赖手工推理启发式方法,便超越了这些结果。值得注意的是,o3 在 2024 年 IOI 上获得金牌,并取得了与顶尖人类选手相当的 Codeforces 评分。总体而言,这些结果表明,扩展通用强化学习而非依赖领域专用技术,为在竞技编程等推理领域实现最先进 AI 提供了一条稳健路径。

英文摘要:

We show that reinforcement learning applied to large language models (LLMs) significantly boosts performance on complex coding and reasoning tasks. Additionally, we compare two general-purpose reasoning models - OpenAI o1 and an early checkpoint of o3 - with a domain-specific system, o1-ioi, which uses hand-engineered inference strategies designed for competing in the 2024 International Olympiad in Informatics (IOI). We competed live at IOI 2024 with o1-ioi and, using hand-crafted test-time strategies, placed in the 49th percentile. Under relaxed competition constraints, o1-ioi achieved a gold medal. However, when evaluating later models such as o3, we find that o3 achieves gold without hand-crafted domain-specific strategies or relaxed constraints. Our findings show that although specialized pipelines such as o1-ioi yield solid improvements, the scaled-up, general-purpose o3 model surpasses those results without relying on hand-crafted inference heuristics. Notably, o3 achieves a gold medal at the 2024 IOI and obtains a Codeforces rating on par with elite human competitors. Overall, these results indicate that scaling general-purpose reinforcement learning, rather than relying on domain-specific techniques, offers a robust path toward state-of-the-art AI in reasoning domains, such as competitive programming.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑