高维数据的快速鲁棒相关
Fast robust correlation for high-dimensional data
摘要：
积矩协方差是多元数据分析的基石，从中可以得到相关性，主成分，马氏距离和许多其他结果。不幸的是，积矩协方差和相应的皮尔逊相关性非常容易受到数据中离群值（异常）的影响。虽然已有一些稳健的协方差度量方法，但很少有适合于目前越来越普遍的高维数据。为此，人们需要这样的方法，其计算与维数成比例，保证产生半正定的协方差阵，并且对离群值足够鲁棒，并且在低变异性的统计意义上足够准确。我们使用数据转换来构建这样的方法。该方法简单、快速、适用范围广。我们通过推导影响函数和故障值，以及计算污染数据的均方误差来研究其鲁棒性。利用这些结果，我们选择了一种整体表现良好的方法。这也使我们能够构建一个更快版本的DetectDeviatingCells方法（Risseeuw和Van den Bossche，2018）来检测细胞离群值，可以处理更高的维度。该方法在具有12，000个变量的基因组数据和具有920，000个维度的彩色视频数据上进行了说明。
英文摘要：
The product moment covariance is a cornerstone of multivariate data analysis, from which one can derive correlations, principal components, Mahalanobis distances and many other results. Unfortunately the product moment covariance and the corresponding Pearson correlation are very susceptible to outliers (anomalies) in the data. Several robust measures of covariance have been developed, but few are suitable for the ultrahigh dimensional data that are becoming more prevalent nowadays. For that one needs methods whose computation scales well with the dimension, are guaranteed to yield a positive semidefinite covariance matrix, and are sufficiently robust to outliers as well as sufficiently accurate in the statistical sense of low variability. We construct such methods using data transformations. The resulting approach is simple, fast and widely applicable. We study its robustness by deriving influence functions and breakdown values, and computing the mean squared error on contaminated data. Using these results we select a method that performs well overall. This also allows us to construct a faster version of the DetectDeviatingCells method (Rousseeuw and Van den Bossche, 2018) to detect cellwise outliers, that can deal with much higher dimensions. The approach is illustrated on genomic data with 12,000 variables and color video data with 920,000 dimensions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
