AI 中文总结
本研究通过分析100亿、1万亿观测的随机数据集及含三个潜在因子的100亿人工构造数据集，证实PCA在极大样本量下稳定且快速收敛，可有效恢复潜在结构，对多领域大规模应用有重要意义。
AI 中文摘要
本研究探究了主成分分析（PCA）应用于观测数量极大的数据集时的表现。尽管统计理论表明，随着样本量增大，抽样误差会减小，样本估计值会收敛到总体值，但关于PCA在百亿或万亿观测规模下的表现，现有的实证证据相对较少。研究分析了三个数据集：100亿观测值的随机数据集、1万亿观测值的随机数据集，以及100亿观测值的人工构造数据集（该数据集被设计为包含三个潜在因子）。结果显示，从100亿随机数据集和1万亿随机数据集得到的PCA解几乎完全一致，表明PCA在极大样本量下具有显著的稳定性。相比之下，人工构造数据集产生了三个主导主成分，它们解释了99.996%的总标准化方差，且成功恢复了预期的潜在因子结构。这些发现表明，PCA解在极大样本量下会快速收敛，且可能在样本量达到万亿之前就已达到实用收敛。这些发现对遥感、数字制图、环境建模等领域的大规模应用具有重要意义，这些领域的数据集通常包含数百万或数十亿观测值。
英文摘要
This study investigated the behavior of Principal Component Analysis (PCA) when applied to datasets with extremely large numbers of observations. Although statistical theory suggests that sampling error diminishes and sample estimates converge toward their population values as sample size increases, relatively little empirical evidence exists regarding the behavior of PCA at scales measured in billions or trillions of observations. Three datasets were analyzed: a 10-billion observation random dataset, a 1-trillion observation random dataset, and a 10-billion observation engineered dataset designed to contain three latent factors. Results showed that the PCA solutions obtained from the 10BillionRandom and 1TrillionRandom datasets were nearly identical, indicating substantial stability of PCA at extremely large sample sizes. In contrast, the engineered dataset produced three dominant principal components that accounted for 99.996% of the total standardized variance and successfully recovered the intended latent-factor structure. These findings suggest that PCA solutions converge rapidly at very large sample sizes and suggest that PCA solutions may reach practical convergence well before sample sizes reach the trillions. These findings have implications for large-scale applications in fields such as remote sensing, digital mapping, environmental modeling, and other domains where datasets routinely contain millions or billions of observations.
Comments12 Pages, 1 Figure, 6 Tables
