CLINB：面向基础模型的气候智能基准
CLINB: A Climate Intelligence Benchmark for Foundational Models
AI总结：
针对大语言模型复杂专业知识处理能力评估难题，本文提出气候领域多模态问答基准CLINB，评测发现前沿模型知识综合能力优异但事实归因存在严重幻觉问题，可为可信科学AI构建提供评测支撑。
摘要：
评估大语言模型（LLMs）处理复杂专业知识的能力仍是一项关键挑战。我们以气候变化领域为切入点，推出CLINB基准，该基准围绕对知识质量和证据支撑有明确要求的开放式、基于事实的多模态问答任务对模型开展评估。CLINB依托由顶尖气候科学家整理的真实用户问题数据集与评估评分规则构建。我们实现并验证了一套基于模型的评估流程，对多款前沿模型开展评测。研究结果揭示了一个关键的二元分化现象：前沿模型展现出出色的知识综合能力，往往具备博士级别的理解水平与表述质量，表现优于由领域专家借助较弱模型整理出的“混合式”答案；但这种表现被事实 grounding 层面的缺陷抵消，其证据质量参差不齐，参考文献和图像存在较高的幻觉率。我们认为，弥合知识综合能力与可验证归因之间的差距，对于在科学工作流中部署AI至关重要，需要CLINB这类可靠、可解释的基准来推动可信AI系统的构建。
英文摘要：
Evaluating how Large Language Models (LLMs) handle complex, specialized knowledge remains a critical challenge. We address this through the lens of climate change by introducing CLINB, a benchmark that assesses models on open-ended, grounded, multimodal question answering tasks with clear requirements for knowledge quality and evidential support. CLINB relies on a dataset of real users' questions and evaluation rubrics curated by leading climate scientists. We implement and validate a model-based evaluation process and evaluate several frontier models. Our findings reveal a critical dichotomy. Frontier models demonstrate remarkable knowledge synthesis capabilities, often exhibiting PhD-level understanding and presentation quality. They outperform "hybrid" answers curated by domain experts assisted by weaker models. However, this performance is countered by failures in grounding. The quality of evidence varies, with substantial hallucination rates for references and images. We argue that bridging this gap between knowledge synthesis and verifiable attribution is essential for the deployment of AI in scientific workflows and that reliable, interpretable benchmarks like CLINB are needed to progress towards building trustworthy AI systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
