计算化学方法的概率性能估计:系统改进概率和排序概率矩阵。I.理论
Probabilistic performance estimators for computational chemistry methods: Systematic Improvement Probability and Ranking Probability Matrix. I. Theory
摘要:
基准误差集的比较是计算化学理论评价的重要工具。由于与误差分布的非正态性和潜在趋势的存在有关的几个原因,按其平均无符号误差对方法进行的标准排序是不令人满意的。最近提出了补充统计量来弥补这些不足,例如绝对误差分布的分位数或平均预测不确定性。我们在这里介绍了一个新的分数,系统改善概率(SIP)的基础上,直接系统明智的比较绝对误差。与所选择的评分规则无关,由于基准数据集的不完整性而导致的统计数据的不确定性也通常被忽视。然而,这种不确定性对于评价排名的稳健性至关重要。在本文中,我们开发了两个基于稳健统计的指标来解决这个问题:P_{inv},两个统计值之间的反转概率,以及\mathbf{P}_{r},排名概率矩阵。我们还证明了这些分数比较中错误集之间的相关性的重要贡献。
英文摘要:
The comparison of benchmark error sets is an essential tool for the evaluation of theories in computational chemistry. The standard ranking of methods by their Mean Unsigned Error is unsatisfactory for several reasons linked to the non-normality of the error distributions and the presence of underlying trends. Complementary statistics have recently been proposed to palliate such deficiencies, such as quantiles of the absolute errors distribution or the mean prediction uncertainty. We introduce here a new score, the systematic improvement probability (SIP), based on the direct system-wise comparison of absolute errors. Independently of the chosen scoring rule, the uncertainty of the statistics due to the incompleteness of the benchmark data sets is also generally overlooked. However, this uncertainty is essential to appreciate the robustness of rankings. In the present article, we develop two indicators based on robust statistics to address this problem: P_{inv}, the inversion probability between two values of a statistic, and \mathbf{P}_{r}, the ranking probability matrix. We demonstrate also the essential contribution of the correlations between error sets in these scores comparisons.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。