PHYSICS：在大学物理问题求解中基准测试基础模型
PHYSICS: Benchmarking Foundation Models on University-Level Physics Problem Solving
AI总结：
本文提出PHYSICS基准测试，用于评估大学水平物理问题求解能力，包含1297个专家标注的问题，涵盖六个核心领域，并通过自动化评估系统揭示了领先基础模型的显著局限性。
摘要：
我们介绍了PHYSICS，一个全面的大学物理问题求解基准测试。它包含1297个专家标注的问题，涵盖六个核心领域：经典力学、量子力学、热力学和统计力学、电磁学、原子物理和光学。每个问题都需要高级物理知识和数学推理。我们开发了一个稳健的自动化评估系统，以实现精确且可靠的验证。对领先基础模型的评估揭示了显著的局限性。即使最先进的模型o3-mini也只能达到59.9%的准确率，突显了解决高水平科学问题的重大挑战。通过全面的错误分析、探索多样的提示策略以及基于检索增强生成（RAG）的知识增强，我们识别出关键的改进领域，为未来的发展奠定了基础。
英文摘要：
We introduce PHYSICS, a comprehensive benchmark for university-level physics problem solving. It contains 1297 expert-annotated problems covering six core areas: classical mechanics, quantum mechanics, thermodynamics and statistical mechanics, electromagnetism, atomic physics, and optics. Each problem requires advanced physics knowledge and mathematical reasoning. We develop a robust automated evaluation system for precise and reliable validation. Our evaluation of leading foundation models reveals substantial limitations. Even the most advanced model, o3-mini, achieves only 59.9% accuracy, highlighting significant challenges in solving high-level scientific problems. Through comprehensive error analysis, exploration of diverse prompting strategies, and Retrieval-Augmented Generation (RAG)-based knowledge augmentation, we identify key areas for improvement, laying the foundation for future advancements.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
