arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.14232cs.LGcs.AIcs.CL
收藏

通过扩展测试时间计算实现IOI金牌奖章的开放权重模型

Scaling Test-Time Compute to Achieve IOI Gold Medal with Open-Weight Models

NVIDIA

Mehrzad Samadi, Aleksander Ficek, Sean Narenthiran, Siddhartha Jain, Wasi Uddin Ahmad, Somshubra Majumdar, Vahid Noroozi, Boris Ginsburg

AI总结:

本文提出GenCluster框架,利用开放权重模型在有限预算下高效探索解决方案空间,实现IOI金牌级性能,缩小开放与闭合系统差距。

摘要:

竞争编程已成为评估大语言模型推理和问题解决能力的严格基准。国际信息学奥林匹克竞赛(IOI)是竞争编程中最受尊敬的年度竞赛之一,已成为比较人类和AI编程能力的关键基准。尽管一些专有模型声称在IOI上达到金牌级别表现,但往往方法不透明,使用开放权重模型实现类似结果仍是个重大挑战。本文提出GenCluster,一个可扩展且可重复的测试时间计算框架,利用开放权重模型达到IOI金牌级性能。它结合大规模生成、行为聚类、排序和轮换提交策略,以在有限验证预算下高效探索多样化的解决方案空间。我们的实验表明,所提出方法的性能与可用计算资源一致,缩小了开放与闭合系统的差距。值得注意的是,我们展示了GenCluster能够首次使用开放权重模型gpt-oss-120b在IOI 2025上获得金牌,为LLM推理的透明和可重复评估设定了新基准。

英文摘要:

Competitive programming has become a rigorous benchmark for evaluating the reasoning and problem-solving capabilities of large language models (LLMs). The International Olympiad in Informatics (IOI) stands out as one of the most prestigious annual competitions in competitive programming and has become a key benchmark for comparing human and AI-level programming ability. While several proprietary models have been claimed to achieve gold medal-level performance at the IOI, often with undisclosed methods, achieving comparable results with open-weight models remains a significant challenge. In this paper, we present GenCluster, a scalable and reproducible test-time compute framework that attains IOI gold-level performance using open-weight models. It combines large-scale generation, behavioral clustering, ranking, and a round-robin submission strategy to efficiently explore diverse solution spaces under limited validation budgets. Our experiments show that the performance of our proposed approach scales consistently with available compute, narrowing the gap between open and closed systems. Notably, we will show that GenCluster can achieve a gold medal at IOI 2025 for the first time with an open-weight model gpt-oss-120b, setting a new benchmark for transparent and reproducible evaluation of reasoning in LLMs

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑