距离相关的卡方检验
The Chi-Square Test of Distance Correlation
AI总结:
本文提出一种非参数、快速的卡方检验用于距离相关,无需置换检验即可估计零分布,具有与置换检验相当的检验功效,并理论上证明了其有效性和一致性。
摘要:
距离相关在数据科学界近来引起了广泛关注:样本统计量计算简单,且当且仅当独立时渐近等于零,这使得它成为在样本量足够时发现任何类型依赖结构的理想选择。一个主要的瓶颈在于检验过程:因为距离相关的零分布依赖于潜在的随机变量和度量选择,通常需要置换检验来估计零分布并计算p值,这对于大量数据来说代价高昂。为克服这一困难,本文提出了一种针对距离相关的卡方检验。在方法上,卡方检验是非参数的,速度极快,适用于使用任何强负类型度量或特征核的偏差校正距离相关。该检验展现出与标准置换检验相似的检验功效,并可用于K样本和偏相关检验。在理论上,我们证明了潜在的卡方分布能很好地近似并支配上尾的极限零分布,证明了卡方检验在检验独立性方面是有效且普遍一致的,并建立了相对于置换检验的检验功效不等式。
英文摘要:
Distance correlation has gained much recent attention in the data science community: the sample statistic is straightforward to compute and asymptotically equals zero if and only if independence, making it an ideal choice to discover any type of dependency structure given sufficient sample size. One major bottleneck is the testing process: because the null distribution of distance correlation depends on the underlying random variables and metric choice, it typically requires a permutation test to estimate the null and compute the p-value, which is very costly for large amount of data. To overcome the difficulty, in this paper we propose a chi-square test for distance correlation. Method-wise, the chi-square test is non-parametric, extremely fast, and applicable to bias-corrected distance correlation using any strong negative type metric or characteristic kernel. The test exhibits a similar testing power as the standard permutation test, and can be utilized for K-sample and partial testing. Theory-wise, we show that the underlying chi-square distribution well approximates and dominates the limiting null distribution in upper tail, prove the chi-square test can be valid and universally consistent for testing independence, and establish a testing power inequality with respect to the permutation test.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。