Multi-Physics：面向中文多学科物理问题的多模态大语言模型推理综合基准
Multi-Physics: A Comprehensive Benchmark for Multimodal LLMs Reasoning on Chinese Multi-Subject Physics Problems
AI总结：
本文提出中文物理推理基准Multi-Physics，含1,412道图像关联题、11个学科和5个难度等级，通过双重评估框架分析20种多模态大语言模型的答案准确率与思维链完整性，并系统研究难度和视觉信息对推理的影响。
摘要：
尽管多模态大语言模型（MLLMs）展现出显著的推理进展，但它们在物理等专业科学领域中的应用揭示了当前评估基准中的明显不足。具体而言，现有基准往往缺乏细粒度的学科覆盖，忽视逐步推理过程，并且主要以英语为中心，未能系统评估视觉信息的作用。为此，我们提出了用于中文物理推理的 Multi-Physics，这是一个包含5个难度等级的综合基准，涵盖1,412道与图像关联的多项选择题，涉及11个高中物理学科。我们采用双重评估框架对20种不同的MLLMs进行评估，同时分析最终答案准确率和其思维链的逐步完整性。此外，我们通过比较改变输入模式前后的模型性能，系统研究了难度等级和视觉信息的影响。我们的工作不仅为社区提供了细粒度资源，还为剖析最先进MLLMs的多模态推理过程提供了稳健的方法论，并且我们的数据集和代码已开源：https://github.com/luozhongze/Multi-Physics。
英文摘要：
While multimodal LLMs (MLLMs) demonstrate remarkable reasoning progress, their application in specialized scientific domains like physics reveals significant gaps in current evaluation benchmarks. Specifically, existing benchmarks often lack fine-grained subject coverage, neglect the step-by-step reasoning process, and are predominantly English-centric, failing to systematically evaluate the role of visual information. Therefore, we introduce \textbf {Multi-Physics} for Chinese physics reasoning, a comprehensive benchmark that includes 5 difficulty levels, featuring 1,412 image-associated, multiple-choice questions spanning 11 high-school physics subjects. We employ a dual evaluation framework to evaluate 20 different MLLMs, analyzing both final answer accuracy and the step-by-step integrity of their chain-of-thought. Furthermore, we systematically study the impact of difficulty level and visual information by comparing the model performance before and after changing the input mode. Our work provides not only a fine-grained resource for the community but also offers a robust methodology for dissecting the multimodal reasoning process of state-of-the-art MLLMs, and our dataset and code have been open-sourced: https://github.com/luozhongze/Multi-Physics.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
