PRL-Bench: 一个全面的基准，评估大语言模型在前沿物理研究中的能力
PRL-Bench: A Comprehensive Benchmark Evaluating LLMs' Capabilities in Frontier Physics Research
AI总结：
PRL-Bench通过理论和计算物理的科研导向评估，系统测试大语言模型执行端到端物理研究的能力，揭示当前LLM在真实科研需求中的不足。
摘要：
代理科学范式要求AI系统进行稳健推理并进行长周期自主探索。然而，当前科学基准仅限于领域知识理解和复杂推理，无法评估真实科研的探索性质和过程复杂性。本文提出理论和计算物理的研究导向评估，作为自然测试场，具备全面领域知识、复杂推理和可验证端到端工作流。我们介绍了PRL-Bench（由LLM进行物理研究），旨在系统映射LLM在执行端到端物理研究中的能力边界。PRL-Bench由100篇最新Physical Review Letters论文构建，并由领域专家验证，涵盖现代物理的五个主要理论和计算密集型子领域：天体物理、凝聚态物理、高能物理、量子信息和统计物理。每个任务均设计为复制真实科研的核心特性，包括探索导向的制定、长周期工作流和客观可验证性，从而重建真实物理研究的推理过程和研究工作流。评估显示，前沿模型的表现仍有限，最佳整体得分低于50，揭示当前LLM能力与真实科研需求之间显著的差距。PRL-Bench为访问下一代AI科学家提供可靠测试场，推动AI系统向自主科学发现迈进。
英文摘要：
The paradigm of agentic science requires AI systems to conduct robust reasoning and engage in long-horizon, autonomous exploration. However, current scientific benchmarks remain confined to domain knowledge comprehension and complex reasoning, failing to evaluate the exploratory nature and procedural complexity of real-world research. In this work, we present research-oriented evaluations in theoretical and computational physics, a natural testbed with comprehensive domain knowledge, complex reasoning, and verifiable end-to-end workflows without reliance on experiments. Here we introduce PRL-Bench (Physics Research by LLMs), a benchmark designed to systematically map the capability boundaries of LLMs in executing end-to-end physics research. Constructed from 100 curated papers from the latest issues of Physical Review Letters since August 2025 and validated by domain experts, PRL-Bench covers five major theory- and computation-intensive subfields of modern physics: astrophysics, condensed matter physics, high-energy physics, quantum information, and statistical physics. Each task in the benchmark is designed to replicate the core properties of authentic scientific research, including exploration-oriented formulation, long-horizon workflows, and objective verifiability, thereby reconstructing the essential reasoning processes and research workflows of real physics research. Evaluation across frontier models shows that performance remains limited, with the best overall score below 50, revealing a pronounced gap between current LLM capabilities and the demands of real scientific research. PRL-Bench serves a reliable testbed for accessing next generation AI scientists advancing AI systems toward autonomous scientific discovery.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
