arXiv 2510.009192026-04-14cs.CLcs.AI

基于检索增强生成在奥林匹克级物理问题求解中的基础模型评估

Benchmarking Foundation Models with Retrieval-Augmented Generation in Olympic-Level Physics Problem Solving

AAII, University of Technology Sydney, New South Wales, Australia（AAII，悉尼大学，新南威尔士州，澳大利亚）；Eindhoven University of Technology, Eindhoven, The Netherlands（埃因霍温理工大学，埃因霍温，荷兰）；University of Liverpool, Liverpool, United Kingdom（利物浦大学，利物浦，英国）；University of New South Wales, New South Wales, Australia（新南威尔士大学，新南威尔士州，澳大利亚）

Shunfeng Zheng, Yudi Zhang, Meng Fang, Zihan Zhang, Zhitan Wu, Mykola Pechenizkiy, Ling Chen

备注：: Accepted to EMNLP 2025 (Findings)

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文通过PhoPile多模态数据集评估检索增强生成在奥林匹克级物理问题求解中的表现，探讨了基础模型结合检索与物理语料对性能提升的影响及存在的挑战。

摘要：

检索增强生成（RAG）结合基础模型在多种任务中表现出色，但其在专家级推理（如奥林匹克级物理问题求解）中的能力仍待探索。受学生通过复习过往问题准备竞赛的启发，我们研究了RAG增强基础模型在物理推理中的潜力。我们引入PhoPile，一个专门用于奥林匹克级物理问题的高质量多模态数据集，以系统研究基于检索的推理。PhoPile包含图表、图形和方程，捕捉了物理问题求解的固有多模态特性。利用PhoPile，我们评估了RAG增强的基础模型，涵盖大型语言模型（LLMs）和大型多模态模型（LMMs）等多种检索器。我们的结果表明，将检索与物理语料结合可以提升模型性能，同时也突显了需要进一步研究的挑战，以推动检索增强的物理推理研究。

英文摘要：

Retrieval-augmented generation (RAG) with foundation models has achieved strong performance across diverse tasks, but their capacity for expert-level reasoning-such as solving Olympiad-level physics problems-remains largely unexplored. Inspired by the way students prepare for competitions by reviewing past problems, we investigate the potential of RAG to enhance physics reasoning in foundation models. We introduce PhoPile, a high-quality multimodal dataset specifically designed for Olympiad-level physics, enabling systematic study of retrieval-based reasoning. PhoPile includes diagrams, graphs, and equations, capturing the inherently multimodal nature of physics problem solving. Using PhoPile, we benchmark RAG-augmented foundation models, covering both large language models (LLMs) and large multimodal models (LMMs) with multiple retrievers. Our results demonstrate that integrating retrieval with physics corpora can improve model performance, while also highlighting challenges that motivate further research in retrieval-augmented physics reasoning.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

AI总结：

摘要：

英文摘要：

相关论文