基于稀疏图码的混合稀疏线性回归学习
Learning Mixtures of Sparse Linear Regressions Using Sparse Graph Codes
摘要：
本文研究了混合稀疏线性回归模型。令$β^{(1)},\ldots,β^{(L)}\in\mathbb{C}^n$是具有全部$ K $非零系数的$ L $未知稀疏参数向量。噪声线性测量以$y_i={x}_i^H β^{(\ell_i)} + w_i$的形式获得，每个测量是从具有未知标签$ \ell_i $的稀疏向量之一随机生成的。我们的目标是以低的样本和计算成本有效地估计参数向量。这个问题提出了重大挑战，因为需要同时解决恢复标签$ \ell_i $的去混合问题以及恢复稀疏向量$ β^{(\ell)} $的估计问题。我们对这个问题的解决方案利用了现代编码理论和统计推断之间的联系。我们介绍了一种新的算法，混合着色，采样的混合物的战略使用查询向量$ {x}_i $构建的思想，从稀疏图代码的基础上。我们的新的代码设计允许有效的去混合和参数估计。在无噪声环境下，对于常数数量的稀疏参数向量，我们的算法实现了$Θ(K)$的阶最优采样和时间复杂度。在存在高斯噪声的情况下，对于具有两个参数向量的问题（即，$L=2$），我们表明，鲁棒混合着色算法实现了接近最佳的$Θ(K polylog(n))$样本和时间复杂度。当$K=O(n^α)$对于某个常数$α\in(0,1)$（即，$K$在$n$中是次线性的），我们可以在环境维度上实现样本复杂度和时间复杂度。在我们的一个实验中，恢复两个回归的混合维数为$n=500$和稀疏$K=50$，我们的算法是$300$倍以上的速度比EM算法，约三分之一的样本成本。
英文摘要：
In this paper, we consider the mixture of sparse linear regressions model. Let $β^{(1)},\ldots,β^{(L)}\in\mathbb{C}^n$ be $ L $ unknown sparse parameter vectors with a total of $ K $ non-zero coefficients. Noisy linear measurements are obtained in the form $y_i={x}_i^H β^{(\ell_i)} + w_i$, each of which is generated randomly from one of the sparse vectors with the label $ \ell_i $ unknown. The goal is to estimate the parameter vectors efficiently with low sample and computational costs. This problem presents significant challenges as one needs to simultaneously solve the demixing problem of recovering the labels $ \ell_i $ as well as the estimation problem of recovering the sparse vectors $ β^{(\ell)} $. Our solution to the problem leverages the connection between modern coding theory and statistical inference. We introduce a new algorithm, Mixed-Coloring, which samples the mixture strategically using query vectors $ {x}_i $ constructed based on ideas from sparse graph codes. Our novel code design allows for both efficient demixing and parameter estimation. In the noiseless setting, for a constant number of sparse parameter vectors, our algorithm achieves the order-optimal sample and time complexities of $Θ(K)$. In the presence of Gaussian noise, for the problem with two parameter vectors (i.e., $L=2$), we show that the Robust Mixed-Coloring algorithm achieves near-optimal $Θ(K polylog(n))$ sample and time complexities. When $K=O(n^α)$ for some constant $α\in(0,1)$ (i.e., $K$ is sublinear in $n$), we can achieve sample and time complexities both sublinear in the ambient dimension. In one of our experiments, to recover a mixture of two regressions with dimension $n=500$ and sparsity $K=50$, our algorithm is more than $300$ times faster than EM algorithm, with about one third of its sample cost.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
