子抽样与刀切法：一种在有限计算资源下进行大规模数据分析的实用便捷方案
Subsampling and Jackknifing: A Practically Convenient Solution for Large Data Analysis with Limited Computational Resources
AI总结：
本文提出一种结合有放回子抽样与刀切去偏的方法，在有限计算资源下高效分析大规模数据，所得最终估计量具有一致性、渐近正态性，且统计效率可与全样本估计量相媲美。
摘要：
现代统计分析经常遇到规模庞大的数据集。对于这些数据集，传统的估计方法往往难以直接使用，因为实践者通常面临有限的计算资源。在大多数情况下，他们并不拥有强大的计算资源（例如 Hadoop 或 Spark）。因此，如何在有限的计算资源下实际分析大规模数据集就成为一个非常重要的问题。为了解决这一问题，本文提出了一种基于子抽样并结合刀切法（jackknifing）的新方法。其核心思想是将整个样本数据视为总体。然后，通过有放回的简单随机抽样方法获得多个规模大幅缩减的子样本。值得注意的是，我们不推荐使用无放回抽样方法，因为这会带来硬盘数据处理上的显著成本；而如果数据在内存中处理，这种成本则不存在。由于子样本数据的规模相对较小，它们可以轻松地整体读入计算机内存，并方便地进行处理。基于这些子样本数据集，可以为目标参数获得刀切去偏（jackknife-debiased）估计量。所得的估计量在统计上具有一致性，且偏差极小。最后，将来自不同子样本的刀切去偏估计量进行平均，以形成最终估计量。我们从理论上证明，最终估计量具有一致性且渐近正态。在非常温和的条件下，其渐近统计效率可以与全样本估计量相媲美。所提出的方法足够简单，易于在大多数实际计算机系统上实现，因此应具有非常广泛的适用性。
英文摘要：
Modern statistical analysis often encounters datasets with large sizes. For these datasets, conventional estimation methods can hardly be used immediately because practitioners often suffer from limited computational resources. In most cases, they do not have powerful computational resources (e.g., Hadoop or Spark). How to practically analyze large datasets with limited computational resources then becomes a problem of great importance. To solve this problem, we propose here a novel subsampling-based method with jackknifing. The key idea is to treat the whole sample data as if they were the population. Then, multiple subsamples with greatly reduced sizes are obtained by the method of simple random sampling with replacement. It is remarkable that we do not recommend sampling methods without replacement because this would incur a significant cost for data processing on the hard drive. Such cost does not exist if the data are processed in memory. Because subsampled data have relatively small sizes, they can be comfortably read into computer memory as a whole and then processed easily. Based on subsampled datasets, jackknife-debiased estimators can be obtained for the target parameter. The resulting estimators are statistically consistent, with an extremely small bias. Finally, the jackknife-debiased estimators from different subsamples are averaged together to form the final estimator. We theoretically show that the final estimator is consistent and asymptotically normal. Its asymptotic statistical efficiency can be as good as that of the whole sample estimator under very mild conditions. The proposed method is simple enough to be easily implemented on most practical computer systems and thus should have very wide applicability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
