arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.27604cs.CVcs.CE
收藏

解码科学实验图像:用于感知、理解和推理的SPUR基准

Decoding Scientific Experimental Images: The SPUR Benchmark for Perception, Understanding, and Reasoning

Beijing University of Posts and Telecommunications(北京邮电大学)
Junpeng Ding, Zichen Tang, Haihong E, Mengyuan Ji, Yang Liu, Haolin Tian, Haiyang Sun, Pengqi Sun, Yang Xu, Yichen Liu, Haocheng Gao, Zijie Xi, Ruomeng Jiang, P… 展开作者

Junpeng Ding, Zichen Tang, Haihong E, Mengyuan Ji, Yang Liu, Haolin Tian, Haiyang Sun, Pengqi Sun, Yang Xu, Yichen Liu, Haocheng Gao, Zijie Xi, Ruomeng Jiang, Peizhi Zhao, Rongjin Li, Yuanze Li, Jiacheng Liu, Zhongjun Yang, Jintong Chen, Siying Lin

AI总结:

提出SPUR基准,通过4264个问答对评估多模态大模型在科学实验图像上的细粒度感知、跨面板关系理解和专家级推理能力,揭示当前模型与专家水平的差距。

摘要:

我们引入了SPUR,一个全面的科学实验图像感知、理解和推理基准,包含来自1084张专家精选图像的4264个问答对。SPUR具有三个关键创新:(1)面板级细粒度感知:评估多模态大语言模型(MLLMs)在六个细粒度面板类型上的三个维度(数值、形态和信息定位)的视觉感知能力;(2)跨面板关系理解:利用平均每样本14.3个面板的复杂图像评估MLLMs解读复杂跨面板关系的能力;(3)专家级推理:跨五个实验范式评估定性和定量推理,以确定模型是否能像人类专家一样从证据中推断结论。对20个MLLMs和四种多模态思维链(MCoT)方法的全面评估表明,当前模型远未达到科学图像解释的专家级要求,凸显了人工智能科学(AI4S)研究的关键瓶颈。

英文摘要:

We introduce SPUR, a comprehensive benchmark for scientific experimental image perception, understanding, and reasoning, comprising 4,264 question-answering (QA) pairs derived from 1,084 expert-curated images. SPUR features three key innovations: (1) Panel-Level Fine-Grained Perception: evaluating the visual perception of multimodal large language models (MLLMs) across three dimensions (numerical, morphological, and information localization) on six fine-grained panel types; (2) Cross-Panel Relation Understanding: utilizing complex images with an average of 14.3 panels per sample to evaluate MLLMs' ability to decipher intricate cross-panel relations; (3) Expert-Level Reasoning: assessment of qualitative and quantitative reasoning across five experimental paradigms to determine if models can infer conclusions from evidence as human experts do. Comprehensive evaluation of 20 MLLMs and four multimodal Chain-of-Thought (MCoT) methods reveals that current models fall significantly short of the expert-level requirements for scientific image interpretation, underscoring a critical bottleneck in AI for Science (AI4S) research.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑