对结果差异的统计显著性进行更准确的检验
More accurate tests for the statistical significance of result differences
摘要:
对召回率、精确率和平衡F分数等指标的差异进行统计显著性测试是经验自然语言处理的必要组成部分。不幸的是,我们在一组实验中发现,许多常用的测试往往低估了显著性,因此不太可能检测到不同技术之间存在的差异。这种低估来自一个经常被违反的独立性假设。我们指出了一些有用的测试,不做这个假设,包括计算密集型随机化测试。
英文摘要:
Statistical significance testing of differences in values of metrics like recall, precision and balanced F-score is a necessary part of empirical natural language processing. Unfortunately, we find in a set of experiments that many commonly used tests often underestimate the significance and so are less likely to detect differences that exist between different techniques. This underestimation comes from an independence assumption that is often violated. We point out some useful tests that do not make this assumption, including computationally-intensive randomization tests.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。