分布式稀疏回归的DECO相关特征空间划分
DECOrrelated feature space partitioning for distributed sparse regression
摘要：
当样本量或数据集的维数很大时，拟合统计模型在计算上是具有挑战性的。缩小问题规模的一种有吸引力的方法是首先将数据集划分为子集，然后使用分布式算法进行拟合。数据集可以水平（在样本空间中）或垂直（在特征空间中）分区。虽然大多数文献集中在样本空间划分，特征空间划分是更有效的时，$p\gg n$。然而，现有的分割特征的方法要么容易受到高相关性的影响，要么在降低模型维度方面效率低下。在本文中，我们解决了这些问题，通过一个新的分布式变量选择和参数估计的并行框架命名为DECO。在DECO中，变量首先被分区并分配给$m$分布式工作者。然后，通过为高维问题设计的任何算法来拟合每个工人中的去相关子集数据。我们表明，通过将去相关步骤，DECO可以实现一致的变量选择和参数估计的每个子集（几乎）没有假设。此外，收敛速度几乎是最小最大最优的稀疏和弱稀疏模型，并不依赖于分区数$m$。大量的数值实验来说明新框架的性能。
英文摘要：
Fitting statistical models is computationally challenging when the sample size or the dimension of the dataset is huge. An attractive approach for down-scaling the problem size is to first partition the dataset into subsets and then fit using distributed algorithms. The dataset can be partitioned either horizontally (in the sample space) or vertically (in the feature space). While the majority of the literature focuses on sample space partitioning, feature space partitioning is more effective when $p\gg n$. Existing methods for partitioning features, however, are either vulnerable to high correlations or inefficient in reducing the model dimension. In this paper, we solve these problems through a new embarrassingly parallel framework named DECO for distributed variable selection and parameter estimation. In DECO, variables are first partitioned and allocated to $m$ distributed workers. The decorrelated subset data within each worker are then fitted via any algorithm designed for high-dimensional problems. We show that by incorporating the decorrelation step, DECO can achieve consistent variable selection and parameter estimation on each subset with (almost) no assumptions. In addition, the convergence rate is nearly minimax optimal for both sparse and weakly sparse models and does NOT depend on the partition number $m$. Extensive numerical experiments are provided to illustrate the performance of the new framework.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
