发表机构
Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China; University of Chinese Academy of Sciences, Beijing, China（中国科学院计算技术研究所; 中国科学院大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究针对MatBench缺乏反映AI预测模型在分布外材料数据上性能能力的问题，结合MatBench流程与OOD性能评估研究，提出自动工具包AutoMatBench，经实验验证其能节省成本并获相似结果，对新材料发现有积极贡献。
AI 中文摘要
材料性能预测（MPP）可从化学成分和结构推断关键性能，加速新型材料的发现和优化。在MPP领域，MatBench是广泛接受的基准测试工具，定义了十多个重要问题并为AI预测模型提供性能评估范式。但它缺乏反映预测模型在分布外（OOD）材料数据上性能的能力，导致新材料发现失败。本研究结合MatBench流程和OOD性能评估的现有研究，实现了大量基准测试配置空间，全面反映各种AI预测模型的性能、能力和缺点。研究发现不同配置值下性能差异巨大，考虑配置对性能结果的因果效应很必要。本研究还提出了具有贝叶斯优化的自动工具包AutoMatBench。实验表明，在十二步优化内，可获得与MatBench和先前OOD研究相似的结果，同时节省一半以上成本。此外，该工具还在MPP基准测试中产生了更重要的发现，对新材料发现的成本和效率有积极贡献。
英文摘要
Material property prediction (MPP) infers key properties from chemical composition and structure, accelerating the discovery and optimization of novel materials. In the realm of MPP, MatBench is a widely accepted benchmarking tool that defines over ten significant problems and provides the paradigm of performance evaluation for AI prediction models. Even though MatBench works well in benchmarking the performances of prediction models on in-distribution (ID) tasks and datasets, it lacks the ability to reflect their performances on out-of-distribution (OOD) material data, resulting failure in new material discovery. By combining the pipelines of MatBench and the existing researches on OOD performance evaluation, this study enables a huge space of benchmarking configurations, comprehensively reflecting the performances, abilities, and disadvantages of various AI prediction models. This work reports that the discrepancy of performances at different configuration values is huge and can be illustrated with prior knowledge and novel insights, therefore consideration of causal effect of configurations on performance results is necessary. In case of the impossibility of enumerative benchmarking at every configuration, this work further proposes AutoMatBench, an automatic toolkit with Bayesian optimization. Experiments with AutoMatBench reports that, within twelve steps of optimization, the similar results with MatBench and former OOD research can be accessed while more than half of the cost are saved. Besides, this tool also yields more essential findings on MPP benchmarking, positively contributing to the cost and efficiency of new material discovery.
