arXiv 2206.080922022-06-16cs.LGstat.ML

关于齐次性及其与线性回归的关系

On the well-spread property and its relation to linear regression

ETH Zürich（苏黎世联邦理工学院）

Hongjie Chen, Tommaso d'Orsi

备注：: To appear in COLT 2022

摘要：

我们考虑稳健的线性回归模型$\boldsymbol{y} = Xβ^* + \boldsymbolη$，其中对手不知道的设计$X \in \mathbb{R}^{n \times d}$可能会选择$\boldsymbolη$损坏所有，但（可能消失）的一部分，观察$\boldsymbol{y}$在一个任意的方式。最近的工作[dLN+21，dNS 21]已经引入了有效的算法，一致恢复的参数向量。这些算法主要依赖于设计矩阵的良好分布（如果矩阵的列跨度远离任何稀疏向量，则矩阵是良好分布的）。在本文中，我们发现，存在一个家庭的设计矩阵缺乏良好的传播，使一致恢复的参数向量在上述稳健线性回归模型是信息理论上是不可能的。我们进一步研究了证明随机矩阵良好扩展性的平均情况下的时间复杂度。我们表明，它是可以有效地证明是否一个给定的$n$ -由- $d$高斯矩阵是良好的传播，如果观察的数量是二次的环境尺寸。我们补充这一结果，显示严格的证据-在对低次多项式的下限的形式-的计算硬度时，观察的数量是$o(d^2)$的这个相同的认证问题。

英文摘要：

We consider the robust linear regression model $\boldsymbol{y} = Xβ^* + \boldsymbolη$, where an adversary oblivious to the design $X \in \mathbb{R}^{n \times d}$ may choose $\boldsymbolη$ to corrupt all but a (possibly vanishing) fraction of the observations $\boldsymbol{y}$ in an arbitrary way. Recent work [dLN+21, dNS21] has introduced efficient algorithms for consistent recovery of the parameter vector. These algorithms crucially rely on the design matrix being well-spread (a matrix is well-spread if its column span is far from any sparse vector). In this paper, we show that there exists a family of design matrices lacking well-spreadness such that consistent recovery of the parameter vector in the above robust linear regression model is information-theoretically impossible. We further investigate the average-case time complexity of certifying well-spreadness of random matrices. We show that it is possible to efficiently certify whether a given $n$-by-$d$ Gaussian matrix is well-spread if the number of observations is quadratic in the ambient dimension. We complement this result by showing rigorous evidence -- in the form of a lower bound against low-degree polynomials -- of the computational hardness of this same certification problem when the number of observations is $o(d^2)$.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

摘要：

英文摘要：

相关论文