当 Tukey 遇见 Chauvenet:一种用于异常值检测的新型箱线图准则
When Tukey meets Chauvenet: a new boxplot criterion for outlier detection
AI总结:
本文融合 Tukey 箱线图与 Chauvenet 准则,提出通过精确控制观测外侧率确定围栏系数的 Chauvenet 型箱线图,兼顾简洁、稳健与样本量适应性,并发布 R 包 ChauBoxplot。
摘要:
由 Tukey(1977)提出的箱须图是描述性统计中最流行的图形方法之一。然而,Tukey 箱线图与样本量无关,因而在异常值检测中产生了所谓“一刀切”的围栏。尽管文献中确实存在关于按样本量调整的箱线图的改进,但其中大多数要么不易实施,要么缺乏理论依据。作为另一种常见的异常值检测规则,Chauvenet 准则使用样本均值和标准差进行检验,但它通常对已包含的异常值敏感,因此不具有稳健性。本文通过结合 Tukey 箱线图与 Chauvenet 准则,提出一种新型箱线图,即 Chauvenet 型箱线图,其围栏系数通过对每个观测值的外侧率进行精确控制来确定。我们的新异常值准则不仅从实际应用角度保持了箱线图的简洁性,而且可作为一种稳健的 Chauvenet 准则。模拟研究以及关于香港公务员薪酬调整的真实数据分析表明,Chauvenet 型箱线图在不同样本量下均表现极佳,因此在实际应用中可被强烈推荐,以替代 Tukey 箱线图和 Chauvenet 准则。最后,为提高该工作的可见度,一个名为 `ChauBoxplot' 的用户友好型 R 包也已在 CRAN 上正式发布。
英文摘要:
The box-and-whisker plot, introduced by Tukey (1977), is one of the most popular graphical methods in descriptive statistics. On the other hand, however, Tukey's boxplot is free of sample size, yielding the so-called "one-size-fits-all" fences for outlier detection. Although improvements on the sample size adjusted boxplots do exist in the literature, most of them are either not easy to implement or lack justification. As another common rule for outlier detection, Chauvenet's criterion uses the sample mean and standard derivation to perform the test, but it is often sensitive to the included outliers and hence is not robust. In this paper, by combining Tukey's boxplot and Chauvenet's criterion, we introduce a new boxplot, namely the Chauvenet-type boxplot, with the fence coefficient determined by an exact control of the outside rate per observation. Our new outlier criterion not only maintains the simplicity of the boxplot from a practical perspective, but also serves as a robust Chauvenet's criterion. Simulation study and a real data analysis on the civil service pay adjustment in Hong Kong demonstrate that the Chauvenet-type boxplot performs extremely well regardless of the sample size, and can therefore be highly recommended for practical use to replace both Tukey's boxplot and Chauvenet's criterion. Lastly, to increase the visibility of the work, a user-friendly R package named `ChauBoxplot' has also been officially released on CRAN.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。