数据驱动材料科学中的鲁棒模型基准和偏差不平衡：MODNet案例研究
Robust model benchmarking and bias-imbalance in data-driven materials science: a case study on MODNet
摘要：
随着材料科学的新型数据驱动方法的数量不断增长，对模型性能进行一致的质量，可靠性和适用性评估至关重要。在本文中，我们对材料最优描述符网络（MODNet）方法和架构进行了基准测试，该方法和架构针对最近发布的MatBench v0.1，一个材料数据集的策划测试套件。MODNet在13个任务中的6个任务上表现优于当前的领导者，同时在另外2个任务上与当前的领导者密切匹配;当样本数量低于10，000时，MODNet表现特别好。在对模型进行基准测试时，要注意两个关注的主题。首先，我们鼓励报告更多样化的指标集，因为它可以对模型性能进行更全面和全面的比较。其次，一个同样重要的任务是对目标域的模型的不确定性评估。可以观察到验证误差的显著变化，这取决于训练集中的不平衡和偏差（即，训练和应用空间之间的相似性）。通过使用集合MODNet模型，可以建立置信区间，并可以量化单个预测的不确定性。不平衡和偏差问题经常被忽视，但对于机器学习在材料科学和凝聚态中的成功应用至关重要。
英文摘要：
As the number of novel data-driven approaches to material science continues to grow, it is crucial to perform consistent quality, reliability and applicability assessments of model performance. In this paper, we benchmark the Materials Optimal Descriptor Network (MODNet) method and architecture against the recently released MatBench v0.1, a curated test suite of materials datasets. MODNet is shown to outperform current leaders on 6 of the 13 tasks, whilst closely matching the current leaders on a further 2 tasks; MODNet performs particularly well when the number of samples is below 10,000. Attention is paid to two topics of concern when benchmarking models. First, we encourage the reporting of a more diverse set of metrics as it leads to a more comprehensive and holistic comparison of model performance. Second, an equally important task is the uncertainty assessment of a model towards a target domain. Significant variations in validation errors can be observed, depending on the imbalance and bias in the training set (i.e., similarity between training and application space). By using an ensemble MODNet model, confidence intervals can be built and the uncertainty on individual predictions can be quantified. Imbalance and bias issues are often overlooked, and yet are important for successful real-world applications of machine learning in materials science and condensed matter.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
