MOF-VERIFY：一种面向MOF假设验证的故障感知智能体框架
MOF-VERIFY: A Failure-Aware Agentic Harness for MOF Hypothesis Verification
浏览论文内容
中文总结 AI 辅助
针对LLM在MOF假设验证中可靠性不明的问题，提出故障感知智能体框架MOF-Verify及四类诊断基准，显著提升验证性能。
中文摘要 AI 辅助
大型语言模型越来越多地被用作AI驱动的材料科学协同科学家中的推理组件，然而由此产生的验证流程的可靠性仍不明确。金属有机框架（MOFs）提供了一个特别具有挑战性的场景，因为结构可能以不同的标识符出现，合成结果强烈依赖于实验条件，证据分布在异构来源中，并且某些假设需要计算而非仅靠文献。我们引入了一个诊断基准，包含四个任务族，涵盖结构锚定、合成条件验证、证据充分性验证以及基于MLIP的计算验证。T-MOF-1至T-MOF-3在闭卷、启用检索和预言机证据设置下进行评估，以定位知识访问、证据获取和推理方面的故障，而T-MOF-4则单独评估计算验证。在这些诊断出的故障模式的指导下，我们开发了MOF-Verify，一种故障感知的智能体框架，在产生最终结论之前针对结构、文献、证据充分性和计算瓶颈进行处理。在多个骨干LLM上，MOF-Verify相较于直接推理和基于检索的基线，显著提高了假设验证性能。基准数据集已在此https URL发布。
英文摘要
Large language models are increasingly used as reasoning components in AI-driven materials Co-Scientists, yet the reliability of the resulting verification pipeline remains unclear. Metal-organic frameworks (MOFs) provide a particularly challenging setting because structures may appear under different identifiers, synthesis outcomes depend strongly on experimental conditions, evidence is distributed across heterogeneous sources, and some hypotheses require computation rather than literature alone. We introduce a diagnostic benchmark with four task families covering structural grounding, synthesis-condition verification, evidence-sufficiency verification, and MLIP-based computational verification. T-MOF-1-3 are evaluated under closed-book, retrieval-enabled, and oracle-evidence settings to localize failures in knowledge access, evidence acquisition, and reasoning, while T-MOF-4 separately evaluates computational verification. Guided by these diagnosed failure modes, we develop MOF-Verify, a failure-aware agentic harness that targets structural, literature, evidence-sufficiency, and computational bottlenecks before producing a final verdict. Across multiple backbone LLMs, MOF-Verify substantially improves hypothesis-verification performance over direct inference and retrieval-based baselines. Benchmark datasets are released at https://github.com/IMMS-Ewha/MOF-Verify-Benchmark.
发表机构
- Institute for Multiscale Matters and System (IMMS), Ewha Womans University（梨花女子大学多尺度物质与系统研究所）
- Lymeric（Lymeric公司）
机构由 AI 辅助整理，请以论文原文为准。
