arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.28039cs.AI
收藏

SpecVQA:一种用于科学图像光谱理解和视觉问答的基准测试

SpecVQA: A Benchmark for Spectral Understanding and Visual Question Answering in Scientific Images

DP Technology(DP技术)

Jialu Shen, Han Lyu, Suyang Zhong, Hanzheng Li, Haoyi Tao, Nan Wang, Changhong Chen, Xi Fang

AI总结:

SpecVQA通过7种典型光谱类型和专家标注的问答对,评估多模态模型在科学光谱理解中的能力,提出光谱数据采样与插值重建方法,提升模型在科学光谱理解中的表现。

摘要:

光谱是一种普遍存在但信息密度高的科学图像形式,对多模态大语言模型(MLLMs)构成挑战,因其结构非标准和领域特定。本文引入SpecVQA,一种专业科学图像基准测试,用于评估多模态模型在科学光谱理解上的能力,涵盖7种代表性光谱类型和专家标注的问答对。目标包括光谱科学问答评估和相应底层任务评估。SpecVQA包含620幅图像和3100个问答对,源自同行评审文献,旨在直接信息提取和领域特定推理。为有效减少token长度同时保持关键曲线特征,我们提出光谱数据采样与插值重建方法。消融研究进一步证实,该方法在所提出基准测试上实现了显著的性能提升。我们测试了 prominent MLLMs 在科学光谱理解上的能力,并展示了排行榜。本工作代表了增强多模态大模型光谱理解的重要一步,并提出了将视觉-语言模型扩展到更广泛科学研究和数据分析的有希望的方向。

英文摘要:

Spectra are a prevalent yet highly information-dense form of scientific imagery, presenting substantial challenges to multimodal large language models (MLLMs) due to their unstructured and domain-specific characteristics. Here we introduce SpecVQA, a professional scientific-image benchmark for evaluating multimodal models on scientific spectral understanding, covering 7 representative spectrum types with expert-annotated question-answer pairs. The aim comprises two aspects: spectra scientific QA evaluation and corresponding underlying task evaluation. SpecVQA contains 620 figures and 3100 QA pairs curated from peer-reviewed literature, targeting both direct information extraction and domain-specific reasoning. To effectively reduce token length while preserving essential curve characteristics, we propose a spectral data sampling and interpolation reconstruction approach. Ablation studies further confirm that the approach achieves substantial performance improvements on the proposed benchmark. We test the capability of prominent MLLMs in scientific spectral understanding on our benchmark and present a leaderboard. This work represents an essential step toward enhancing spectral understanding in multimodal large models and suggests promising directions for extending visual-language models to broader scientific research and data analysis.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑