arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2304.03019math.STstat.COstat.MEstat.TH
收藏

最优子抽样设计

Optimal subsampling designs

Henrik Imberg, Marina Axelson-Fisk, Johan Jonasson

AI总结:

针对海量数据子抽样缺乏统一最优设计理论的问题,提出涵盖有限总体推断与回归建模的通用最优设计理论,并构建兼具计算高效与不变性的线性最优性准则,在大幅降低计算需求的同时保持高 D-efficiency。

摘要:

子抽样通常用于克服在分析有限总体和海量数据集时面临的计算与经济瓶颈。现有方法通常适用范围有限,并且使用了存在已知缺陷的最优性准则(例如 A-optimality),例如缺乏对数据测量尺度和模型参数化的不变性。目前仍缺乏关于最优子抽样设计的统一理论。我们提出了一种针对一般数据子抽样问题的最优设计理论,包括有限总体推断、参数密度估计和回归建模。我们的理论涵盖并推广了基于不等概率抽样和逆概率加权的最优子数据选择领域的大多数现有方法。我们推导出了一般类最优性准则的最优性条件,并提出了在 Poisson 和 multinomial 抽样设计下寻找最优抽样方案的相应算法。我们提出了一类新的具有变换和参数化不变性的线性最优性准则,它兼具两者的优点:A-optimality 的计算可处理性和类似于 D-optimality 的不变性属性。该方法在交通安全领域的应用中得到了说明。在我们的实验中,所提出的不变性线性最优性准则实现了 92-99% 的 D-efficiency,同时计算需求降低了 90-95%。相比之下,A-optimality 准则在其中两个示例中仅分别实现了 46% 和 60% 的 D-efficiency。

英文摘要:

Subsampling is commonly used to overcome computational and economical bottlenecks in the analysis of finite populations and massive datasets. Existing methods are often limited in scope and use optimality criteria (e.g., A-optimality) with well-known deficiencies, such as lack of invariance to the measurement-scale of the data and parameterisation of the model. A unified theory of optimal subsampling design is still lacking. We present a theory of optimal design for general data subsampling problems, including finite population inference, parametric density estimation, and regression modelling. Our theory encompasses and generalises most existing methods in the field of optimal subdata selection based on unequal probability sampling and inverse probability weighting. We derive optimality conditions for a general class of optimality criteria, and present corresponding algorithms for finding optimal sampling schemes under Poisson and multinomial sampling designs. We present a novel class of transformation- and parameterisation-invariant linear optimality criteria which enjoy the best of two worlds: the computational tractability of A-optimality and invariance properties similar to D-optimality. The methodology is illustrated on an application in the traffic safety domain. In our experiments, the proposed invariant linear optimality criteria achieve 92-99% D-efficiency with 90-95% lower computational demand. In contrast, the A-optimality criterion has only 46% and 60% D-efficiency on two of the examples.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑