利用局部错误发现率识别大规模高维数据集中的完整相关结构
Identifying the Complete Correlation Structure in Large-Scale High-Dimensional Data Sets with Local False Discovery Rates
AI总结：
本文提出一种基于样本相干矩阵、自助法和局部错误发现率的新方法，在不假设先验相关结构的前提下，对高维少样本多数据集进行双重错误发现率控制，从而识别完整相关结构，并能应对分布不确定性、重尾噪声和离群值。
摘要：
识别多个数据集中依赖成分是许多实际应用中的基本问题。这些应用中的挑战在于，数据集通常是高维的，观测或可用样本很少，并且包含概率分布未知的潜在成分。本文提出了该问题的一种新数学表述，能够在严格的假阳性控制下推断潜在的相关结构。特别是，错误发现率同时在两个层面上被控制在预定义阈值。所使用的检验统计量来源于样本相干矩阵。所需的概率模型通过自助法（bootstrap）从数据中学习得到。局部错误发现率被用于解决多重假设检验问题。与文献中现有技术相比，所提出的技术不假设先验相关结构，并且在数据集数量大而观测数量少的情况下表现良好。此外，它能够处理分布不确定性、重尾噪声和离群值的存在。
英文摘要：
The identification of the dependent components in multiple data sets is a fundamental problem in many practical applications. The challenge in these applications is that often the data sets are high-dimensional with few observations or available samples and contain latent components with unknown probability distributions. A novel mathematical formulation of this problem is proposed, which enables the inference of the underlying correlation structure with strict false positive control. In particular, the false discovery rate is controlled at a pre-defined threshold on two levels simultaneously. The deployed test statistics originate in the sample coherence matrix. The required probability models are learned from the data using the bootstrap. Local false discovery rates are used to solve the multiple hypothesis testing problem. Compared to the existing techniques in the literature, the developed technique does not assume an a priori correlation structure and work well when the number of data sets is large while the number of observations is small. In addition, it can handle the presence of distributional uncertainties, heavy-tailed noise, and outliers.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
