前沿科学：评估人工智能执行专家级科学任务的能力
FrontierScience: Evaluating AI's Ability to Perform Expert-Level Scientific Tasks
AI总结：
FrontierScience是一个评估人工智能执行专家级科学任务能力的基准，通过奥林匹克和研究两个赛道，涵盖多个科学领域，采用细致的评分框架评估模型能力。
摘要：
我们介绍了FrontierScience，一个评估前沿语言模型在专家级科学推理中的能力的基准。最近的模型进展几乎饱和了现有的科学基准，这些基准通常依赖于多项选择知识问题或已发表的信息。FrontierScience通过两个互补的赛道来解决这一差距：（1）奥林匹克，包含国际奥林匹克问题，水平相当于IPhO、IChO和IBO；（2）研究，包含博士级别的开放式问题，代表科学研究中的子任务。FrontierScience包含数百个问题（包括160个在开源的金牌集中的问题），涵盖物理学、化学和生物学的子领域，从量子电动力学到合成有机化学。所有奥林匹克问题都是由国际奥林匹克奖牌得主和国家团队教练原创的，以确保难度、原创性和事实性的标准。所有研究问题都是由博士科学家（博士生、博士后研究员或教授）编写并验证的。对于研究部分，我们引入了一个粒度化的基于评分标准的评估框架，以评估模型在解决研究任务过程中的能力，而不是仅判断一个独立的最终答案。
英文摘要：
We introduce FrontierScience, a benchmark evaluating expert-level scientific reasoning in frontier language models. Recent model progress has nearly saturated existing science benchmarks, which often rely on multiple-choice knowledge questions or already published information. FrontierScience addresses this gap through two complementary tracks: (1) Olympiad, consisting of international olympiad problems at the level of IPhO, IChO, and IBO, and (2) Research, consisting of PhD-level, open-ended problems representative of sub-tasks in scientific research. FrontierScience contains several hundred questions (including 160 in the open-sourced gold set) covering subfields across physics, chemistry, and biology, from quantum electrodynamics to synthetic organic chemistry. All Olympiad problems are originally produced by international Olympiad medalists and national team coaches to ensure standards of difficulty, originality, and factuality. All Research problems are research sub-tasks written and verified by PhD scientists (doctoral candidates, postdoctoral researchers, or professors). For Research, we introduce a granular rubric-based evaluation framework to assess model capabilities throughout the process of solving a research task, rather than judging only a standalone final answer.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
