科学边缘评估：SEE——迈向真实科学发现的缺失环节

中文摘要 AI 辅助

大型语言模型（LLMs）正越来越多地参与科学发现，但目前尚不清楚它们是否能支持复杂的真实实验室科学研究。本文介绍Science Edge Evaluation（SEE），这是一个多模态基准，包含由专家精心设计的问题，这些问题基于同行评审的文献以及化学、生物学和材料科学的实验实践。对19个多模态大型语言模型（MLLMs）的评估显示，即使是表现最佳的模型也仅达到48.7%的准确率。此外，通用模型的平均表现优于科学专用模型。在视觉智能体评估中，工具的使用将最佳准确率提高到52.7%。工具使用可以扩展模型可获取的信息，但更多的信息并不一定能带来可靠的科学推理。关键挑战在于模型能否在原始实验证据的范围内管理工具生成的信息。这些发现共同表明，当前的MLLMs仍然无法从实验结果中做出可靠、合理且受证据约束的推断，而这是真实科学发现中的一项基本能力。弥合这一差距需要MLLMs从解释已确立的科学概念转变为从实验数据中推导新颖且基于证据的见解。

英文摘要

Large language models (LLMs) are increasingly involved in scientific discovery, yet it remains unclear whether they can support complex real laboratory science. Here we introduce Science Edge Evaluation (SEE), a multimodal benchmark of expert-curated questions grounded in peer-reviewed literature and experimental practice in chemistry, biology, and materials science. Evaluation of 19 multimodal large language models (MLLMs) shows that even the best-performing model reaches only 48.7% accuracy. Moreover, general-purpose models outperform science-specialized models on average. In the visual-agent evaluation, the use of tools increases the best accuracy to 52.7%. Tool use can expand the information available to models, but more information does not necessarily lead to reliable scientific reasoning. The key challenge is whether models can manage tool-derived information within the boundaries of the original experimental evidence. Together, these findings reveal that current MLLMs still cannot reliably make justified and evidence-bounded inferences from experimental results, which is an essential capability in real scientific discovery. Bridging this gap requires MLLMs to transition from explaining established scientific concepts to deriving novel and evidence-based insights from experimental data.

发表机构

• Alibaba Group（阿里巴巴集团）
• University of Chinese Academy of Sciences（中国科学院大学）
• Tsinghua University（清华大学）
• University of Alberta（阿尔伯塔大学）
• Zhejiang University（浙江大学）

机构由 AI 辅助整理，请以论文原文为准。

相关深度报道

• 阿里通义SEE给19个多模态模型出实验题，最强准确率也没过50%