ATLAS：面向前沿科学推理的高难度、多学科基准
ATLAS: A High-Difficulty, Multidisciplinary Benchmark for Frontier Scientific Reasoning
AI总结：
ATLAS是一个由领域专家开发的高难度、跨学科科学推理基准，通过原创问题和严格质量控制，评估模型在多领域知识整合和复杂推理能力上的表现。
摘要：
大型语言模型（LLMs）的快速发展导致其在许多现有基准上的性能趋于饱和，质疑了其区分前沿模型的能力。同时，现有高难度基准往往存在学科聚焦狭窄、答案格式过于简化以及易受数据污染的问题，与真实世界科学探究存在可信度差距。为解决这些问题，我们引入ATLAS（面向通用人工智能的科学逻辑应用测试平台），由约800个原创问题组成的大型、高难度且跨学科评估套件。由领域专家（博士及以上水平）开发，涵盖数学、物理、化学、生物学、计算机科学、地球科学和材料科学七个核心科学领域。其关键特征包括：（1）高原创性和抗污染性，所有问题均为原创或大幅改编以防止测试数据泄漏；（2）跨学科聚焦，旨在评估模型在科学领域间整合知识和推理的能力；（3）高保真答案，优先考虑涉及多步骤推理和LaTeX格式表达的复杂、开放式答案，而非简单的单选题；（4）严格质量控制，采用专家同行评审和对抗性测试的多阶段流程，确保问题难度、科学价值和正确性。我们还提出了一种稳健的评估范式，利用LLM评委面板进行自动化、细致的复杂答案评估。初步结果表明，ATLAS在领先模型上的有效性在区分其高级科学推理能力方面表现突出。我们计划将ATLAS发展为一个长期、开放、社区驱动的平台，为迈向人工通用智能提供可靠的
英文摘要：
The rapid advancement of Large Language Models (LLMs) has led to performance saturation on many established benchmarks, questioning their ability to distinguish frontier models. Concurrently, existing high-difficulty benchmarks often suffer from narrow disciplinary focus, oversimplified answer formats, and vulnerability to data contamination, creating a fidelity gap with real-world scientific inquiry. To address these challenges, we introduce ATLAS (AGI-Oriented Testbed for Logical Application in Science), a large-scale, high-difficulty, and cross-disciplinary evaluation suite composed of approximately 800 original problems. Developed by domain experts (PhD-level and above), ATLAS spans seven core scientific fields: mathematics, physics, chemistry, biology, computer science, earth science, and materials science. Its key features include: (1) High Originality and Contamination Resistance, with all questions newly created or substantially adapted to prevent test data leakage; (2) Cross-Disciplinary Focus, designed to assess models' ability to integrate knowledge and reason across scientific domains; (3) High-Fidelity Answers, prioritizing complex, open-ended answers involving multi-step reasoning and LaTeX-formatted expressions over simple multiple-choice questions; and (4) Rigorous Quality Control, employing a multi-stage process of expert peer review and adversarial testing to ensure question difficulty, scientific value, and correctness. We also propose a robust evaluation paradigm using a panel of LLM judges for automated, nuanced assessment of complex answers. Preliminary results on leading models demonstrate ATLAS's effectiveness in differentiating their advanced scientific reasoning capabilities. We plan to develop ATLAS into a long-term, open, community-driven platform to provide a reliable "ruler" for progress toward Artificial General Intelligence.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
