利用大语言模型进行纳米合成机理解释：坚实基础还是纯粹猜想？
Leveraging large language models for nano synthesis mechanism explanation: solid foundations or mere conjectures?
AI总结：
本研究针对现有大语言模型科学评估缺乏机制推理考察的问题，构建金纳米颗粒合成机制选择题基准，提出基于置信度的c-score评估指标，实验证实大语言模型确实理解相关物理化学机制而非靠猜想，为科学AI工具开发提供支撑。
摘要：
随着人工智能（AI）的快速发展，GPT-4等大语言模型（LLM）在科学界受到广泛关注，展现出推动科学发现的巨大潜力。这一进展提出了一个关键问题：这些LLM是否与真实的物理化学原理良好对齐？当前的评估策略主要强调基于事实的知识，例如材料性能预测或名称识别，但往往缺乏对需要逻辑推理的基础物理化学机制的理解。为弥补这一差距，本研究构建了一个包含775道选择题的基准测试，聚焦于金纳米颗粒的合成机制。通过反思现有评估指标，我们质疑直接的对错评估是否仅能体现猜想。因此，我们提出了一种新的评估指标——基于置信度的分数（c-score），该指标通过探测输出logits来推导正确答案的精确概率。基于大量实验，结果表明在金纳米颗粒合成的语境下，LLM能够理解其背后的物理化学机制，而非依赖猜想。本研究凸显了LLM掌握内在科学机制的潜力，并为在各科学领域开发更可靠、有效的AI工具奠定了基础。
英文摘要：
With the rapid development of artificial intelligence (AI), large language models (LLMs) such as GPT-4 have garnered significant attention in the scientific community, demonstrating great potential in advancing scientific discovery. This progress raises a critical question: are these LLMs well-aligned with real-world physicochemical principles? Current evaluation strategies largely emphasize fact-based knowledge, such as material property prediction or name recognition, but they often lack an understanding of fundamental physicochemical mechanisms that require logical reasoning. To bridge this gap, our study developed a benchmark consisting of 775 multiple-choice questions focusing on the mechanisms of gold nanoparticle synthesis. By reflecting on existing evaluation metrics, we question whether a direct true-or-false assessment merely suggests conjecture. Hence, we propose a novel evaluation metric, the confidence-based score (c-score), which probes the output logits to derive the precise probability for the correct answer. Based on extensive experiments, our results show that in the context of gold nanoparticle synthesis, LLMs understand the underlying physicochemical mechanisms rather than relying on conjecture. This study underscores the potential of LLMs to grasp intrinsic scientific mechanisms and sets the stage for developing more reliable and effective AI tools across various scientific domains.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
