AI 中文总结
该研究针对科学智能体幻觉的拓扑结构评估缺口，提出SCHEMA框架，通过构建科学概念图等方式评估，发现幻觉集中于知识枢纽、最终准确率与推理轨迹诚实性脱钩，为高风险科学应用提供机制级评估方案。
AI 中文摘要
大型语言模型（LLM）智能体正越来越多地被部署到科学研究中，在这类场景里可靠性至关重要，且底层知识呈密集互联状态。在此环境下，幻觉的危害尤为严重：关于基础概念的单个错误主张会通过多步推理传播，进而破坏整个推理轨迹。现有的幻觉基准大多在表层层面运作，孤立地对待事实，依赖忽略这种拓扑结构的统一准确率指标。我们针对这一缺口提出SCHEMA，这是首个基于证据、面向科学智能体幻觉的拓扑感知评估框架。SCHEMA可从基准种子与文献证据自动构建科学概念图，综合生成涵盖主张验证、多跳推理、开放式解释及实验代码生成的图基任务，并通过两种互补诊断方式评估智能体：轨迹幻觉流水线通过拓扑加权严重程度评分大规模审计中间推理过程，而多智能体反事实归因模块则能精准定位选定失败案例背后的因果机制。SCHEMA的结果显示，幻觉集中于少数高度连通的知识枢纽，且最终答案准确率与轨迹诚实性相互脱钩；模型常通过结构有缺陷的推理得出正确结论。这些结果表明，对于高风险科学应用，仅终端准确率不足以作为智能体可靠性的信号，需开展基于知识拓扑的机制层面评估。代码可在此URL获取。
英文摘要
Large language model (LLM) agents are increasingly deployed in scientific research, where reliability is critical and the underlying knowledge is densely interconnected. In such settings, hallucinations are particularly damaging: a single erroneous claim on a foundational concept can propagate through multi-step reasoning and corrupt entire trajectories. Existing hallucination benchmarks largely operate at the surface level, treating facts in isolation and relying on uniform accuracy metrics that ignore this topological structure. We address this gap with SCHEMA, the first evidence-grounded, topology-aware evaluation framework for hallucinations in scientific agents. SCHEMA automatically constructs scientific concept graphs from benchmark seeds and literature evidence, synthesizes graph-grounded tasks spanning claim verification, multi-hop reasoning, open-ended explanation, and experimental code generation, and evaluates agents with two complementary diagnostics. A trajectory hallucination pipeline audits intermediate reasoning at scale via a topology-weighted severity score, while a multi-agent counterfactual attribution module pinpoints the causal mechanism behind selected failures. SCHEMA reveals that hallucinations concentrate at a small set of highly connected knowledge hubs, and that final-answer accuracy decouples from trajectory honesty; models often reach correct conclusions through structurally flawed reasoning. These results indicate that for high-stakes scientific applications, terminal accuracy alone is an insufficient signal of agent reliability, motivating mechanism-level evaluation grounded in knowledge topology. Code is available at https://github.com/circles-post/SCHEMA.
Comments36 pages, 7 figures and 5 tables
