arXiv 2103.070202021-03stat.MLcs.ITcs.LGmath.ITmath.STstat.TH

通过凸规划进行最大线性回归

Max-Linear Regression by Convex Programming

The Ohio State University（俄亥俄州立大学）；Georgia Institute of Technology（佐治亚理工学院）

Seonho Kim, Sohail Bahmani, Kiryung Lee

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出锚定回归（AR）凸规划方法解决多元最大线性回归的参数估计问题，证明在高斯观测下以高概率恢复参数，并给出精确恢复所需样本复杂度为 $k^{4}p$，实验表明该方法与交替最小化算法具有竞争力且递归应用可进一步提升精度。

摘要：

我们考虑多元最大线性回归问题，其中模型参数 $\boldsymbolβ_{1},\dotsc,\boldsymbolβ_{k}\in\mathbb{R}^{p}$ 需要从 $n$ 个独立（含噪）观测样本 $y = \max_{1\leq j \leq k} \boldsymbolβ_{j}^{\mathsf{T}} \boldsymbol{x} + \mathrm{noise}$ 中估计。最大线性模型极大地推广了传统线性模型，并且当线性模型数量 $k$ 足够大时，它可以以任意精度逼近任何凸函数。然而，最大线性模型固有的非线性使得回归参数的估计在计算上具有挑战性。特别是，文献中尚不存在基于凸规划的估计器。我们提出并分析了一个可扩展的凸规划，即锚定回归（AR），作为最大线性回归问题的估计器。在标准高斯观测设置下，我们给出了非渐近性能保证，表明该凸规划以高概率恢复参数。当 $k$ 个线性分量等可能地取得最大值时，我们的结果表明，在忽略对数因子的情况下，精确恢复所需的无噪声观测数量为 $k^{4}p$ 量级。这一样本复杂度与交替最小化方法（Ghosh 等人，2021）的结果一致。此外，当观测值受到任意确定性噪声污染时，同样的样本复杂度仍然成立。我们提供的实验结果表明，我们的方法的表现与理论预测一致，并且与交替最小化算法相比具有竞争力，尤其是在存在乘性伯努利噪声的情况下。此外，我们还通过实验表明，对 AR 进行递归应用可以显著提高估计精度。

英文摘要：

We consider the multivariate max-linear regression problem where the model parameters $\boldsymbolβ_{1},\dotsc,\boldsymbolβ_{k}\in\mathbb{R}^{p}$ need to be estimated from $n$ independent samples of the (noisy) observations $y = \max_{1\leq j \leq k} \boldsymbolβ_{j}^{\mathsf{T}} \boldsymbol{x} + \mathrm{noise}$. The max-linear model vastly generalizes the conventional linear model, and it can approximate any convex function to an arbitrary accuracy when the number of linear models $k$ is large enough. However, the inherent nonlinearity of the max-linear model renders the estimation of the regression parameters computationally challenging. Particularly, no estimator based on convex programming is known in the literature. We formulate and analyze a scalable convex program given by anchored regression (AR) as the estimator for the max-linear regression problem. Under the standard Gaussian observation setting, we present a non-asymptotic performance guarantee showing that the convex program recovers the parameters with high probability. When the $k$ linear components are equally likely to achieve the maximum, our result shows a sufficient number of noise-free observations for exact recovery scales as {$k^{4}p$} up to a logarithmic factor. { This sample complexity coincides with that by alternating minimization (Ghosh et al., {2021}). Moreover, the same sample complexity applies when the observations are corrupted with arbitrary deterministic noise. We provide empirical results that show that our method performs as our theoretical result predicts, and is competitive with the alternating minimization algorithm particularly in presence of multiplicative Bernoulli noise. Furthermore, we also show empirically that a recursive application of AR can significantly improve the estimation accuracy.}

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

AI总结：

摘要：

英文摘要：

相关论文