PhysicsEval：提升大型语言模型在物理问题上推理能力的推理时技术
PhysicsEval: Inference-Time Techniques to Improve the Reasoning Proficiency of Large Language Models on Physics Problems
AI总结：
本文评估了前沿LLM解决物理问题的性能，采用多种推理时技术与多智能体框架显著提升了模型表现，并发布了包含19,609个问题的新基准PHYSICSEVAL。
摘要：
物理学作为人类智慧的基石，推动着技术的演进并加深了我们对宇宙基本原理的理解。当代文献中包含一些以解决物理问题为核心的工作，这是自然语言推理的一个关键领域。在本文中，我们评估了前沿LLM在解决数学型和描述型物理问题上的性能。我们还采用了大量的推理时技术和智能体框架来提升模型性能。这包括由其他较小的LLM智能体以累积方式验证所提出的解决方案，并对这些技术所带来的性能进行了比较分析。当将多智能体框架应用于模型初始表现较差的问题时，出现了显著的性能提升。此外，我们引入了一个新的物理问题评估基准PHYSICSEVAL，包含从各物理教材中提取的19,609个问题，以及从物理论坛和教育网站抓取的相应正确解答。我们的代码和数据已在https://github.com/areebuzair/PhysicsEval上公开。
英文摘要：
The discipline of physics stands as a cornerstone of human intellect, driving the evolution of technology and deepening our understanding of the fundamental principles of the cosmos. Contemporary literature includes some works centered on the task of solving physics problems - a crucial domain of natural language reasoning. In this paper, we evaluate the performance of frontier LLMs in solving physics problems, both mathematical and descriptive. We also employ a plethora of inference-time techniques and agentic frameworks to improve the performance of the models. This includes the verification of proposed solutions in a cumulative fashion by other, smaller LLM agents, and we perform a comparative analysis of the performance that the techniques entail. There are significant improvements when the multi-agent framework is applied to problems that the models initially perform poorly on. Furthermore, we introduce a new evaluation benchmark for physics problems, ${\rm P{\small HYSICS}E{\small VAL}}$, consisting of 19,609 problems sourced from various physics textbooks and their corresponding correct solutions scraped from physics forums and educational websites. Our code and data are publicly available at https://github.com/areebuzair/PhysicsEval.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
