通过大规模合成数据集与专家级LLM-as-a-Judge实现全自动材料发现
Towards Fully-Automated Materials Discovery via Large-Scale Synthesis Dataset and Expert-Level LLM-as-a-Judge
AI总结：
本研究构建了含1.7万条专家验证合成配方的AlchemyBench基准，提出LLM-as-a-Judge评估框架，为大语言模型在材料合成预测中的应用提供支撑，助力高效实验设计与材料创新加速。
摘要：
材料合成对于储能、催化、电子学和生物医学设备等领域的创新至关重要。然而，该过程高度依赖由专家直觉指导的经验性试错方法。我们的工作旨在通过提供实用的、数据驱动的资源为材料科学界提供支持。我们从开放获取文献中整理了一个包含1.7万条经专家验证的合成配方的综合数据集，该数据集构成了我们新开发的基准AlchemyBench的基础。AlchemyBench提供了一个端到端框架，支持大语言模型应用于合成预测的研究，涵盖的关键任务包括原材料与设备预测、合成流程生成以及表征结果预测。我们提出了一种LLM-as-a-Judge框架，利用大语言模型进行自动化评估，证明其与专家评估具有很强的统计一致性。总体而言，我们的贡献为探索大语言模型在预测和指导材料合成方面的能力提供了支撑基础，最终为更高效的实验设计和材料科学领域的加速创新铺平了道路。
英文摘要：
Materials synthesis is vital for innovations such as energy storage, catalysis, electronics, and biomedical devices. Yet, the process relies heavily on empirical, trial-and-error methods guided by expert intuition. Our work aims to support the materials science community by providing a practical, data-driven resource. We have curated a comprehensive dataset of 17K expert-verified synthesis recipes from open-access literature, which forms the basis of our newly developed benchmark, AlchemyBench. AlchemyBench offers an end-to-end framework that supports research in large language models applied to synthesis prediction. It encompasses key tasks, including raw materials and equipment prediction, synthesis procedure generation, and characterization outcome forecasting. We propose an LLM-as-a-Judge framework that leverages large language models for automated evaluation, demonstrating strong statistical agreement with expert assessments. Overall, our contributions offer a supportive foundation for exploring the capabilities of LLMs in predicting and guiding materials synthesis, ultimately paving the way for more efficient experimental design and accelerated innovation in materials science.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
