arXiv 2511.146312025-11-18cs.CLcs.AIcs.CVcs.MA

利用视觉-语言模型能力增强智能体自主科学发现

Enhancing Agentic Autonomous Scientific Discovery with Vision-Language Model Capabilities

Department of Physics, University of Cambridge, Cambridge, United Kingdom（剑桥大学物理系）；Kavli Institute for Cosmology, University of Cambridge, Cambridge, United Kingdom（剑桥大学卡弗利天文研究所）；Department of Physics and Astronomy, Haverford College, 370 Lancaster Avenue, Haverford, PA 19041, USA（哈弗福德学院物理与天文学系）；Division of Physics, Mathematics and Astronomy, California Institute of Technology, Pasadena, CA 91125, USA（加州理工学院物理、数学与天文学系）；Institute of Astronomy, University of Cambridge, Cambridge, United Kingdom（剑桥大学天文研究所）

Kahaan Gandhi, Boris Bolliet, Inigo Zubeldia

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

该研究提出用视觉-语言模型（VLMs）引导多智能体系统，通过VLM评判图表并动态生成领域标准，实现自主科学发现中错误纠正与实时数据分析引导，在基准任务中性能优于纯代码和代码加文本基线。

摘要：

我们表明，由视觉-语言模型（VLMs）引导的多智能体系统可改进端到端自主科学发现。通过将图表视为可验证检查点，作为评判者的VLM根据动态生成的领域特定标准评估图形，使智能体能够纠正自身错误并实时引导探索性数据分析。宇宙学和天体化学领域的案例研究显示，系统可在无人干预的情况下从错误推理路径恢复并适应新数据集。在10项数据驱动发现基准任务中，VLM增强系统的“一次通过”得分达0.7-0.8，而纯代码基线为0.2-0.3、代码加文本基线为0.4-0.5，同时提供可审计的推理轨迹以提升可解释性。

英文摘要：

We show that multi-agent systems guided by vision-language models (VLMs) improve end-to-end autonomous scientific discovery. By treating plots as verifiable checkpoints, a VLM-as-a-judge evaluates figures against dynamically generated domain-specific rubrics, enabling agents to correct their own errors and steer exploratory data analysis in real-time. Case studies in cosmology and astrochemistry demonstrate recovery from faulty reasoning paths and adaptation to new datasets without human intervention. On a 10-task benchmark for data-driven discovery, VLM-augmented systems achieve pass at 1 scores of 0.7-0.8, compared to 0.2-0.3 for code-only and 0.4-0.5 for code-and-text baselines, while also providing auditable reasoning traces that improve interpretability. Code available here: https://github.com/CMBAgents/cmbagent

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。