arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1801.03305physics.chem-phphysics.data-an
收藏

计算化学方法的概率性能估计:绝对误差的经验累积分布函数

Probabilistic performance estimators for computational chemistry methods: the empirical cumulative distribution function of absolute errors

Pascal Pernot, Andreas Savin

摘要:

计算化学中的基准研究使用参考数据集通过误差统计来评估方法的准确性。常用的误差统计,如平均有符号误差和平均无符号误差,并不告知最终用户这些方法所附带的预测误差的预期幅度。我们发现,模型误差的分布既不是正态分布,也不是零中心分布,这些误差统计量不能用来推断预测误差概率。为了克服这一限制,我们提倡使用更多的信息统计,基于经验累积分布函数的无符号误差,即(1)概率为一个新的计算有一个绝对误差低于选定的阈值,(2)最大幅度的错误,可以预期与选定的高置信水平。这些统计数据也被证明非常适合基准和排名研究。此外,所有基准测试统计数据的标准误差取决于参考数据集的大小。系统地公布这些标准误差将非常有助于评估基准结论的统计可靠性。

英文摘要:

Benchmarking studies in computational chemistry use reference datasets to assess the accuracy of a method through error statistics. The commonly used error statistics, such as the mean signed and mean unsigned errors, do not inform end-users on the expected amplitude of prediction errors attached to these methods. We show that, the distributions of model errors being neither normal nor zero-centered, these error statistics cannot be used to infer prediction error probabilities. To overcome this limitation, we advocate for the use of more informative statistics, based on the empirical cumulative distribution function of unsigned errors, namely (1) the probability for a new calculation to have an absolute error below a chosen threshold, and (2) the maximal amplitude of errors one can expect with a chosen high confidence level. Those statistics are also shown to be well suited for benchmarking and ranking studies. Moreover, the standard error on all benchmarking statistics depends on the size of the reference dataset. Systematic publication of these standard errors would be very helpful to assess the statistical reliability of benchmarking conclusions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑