arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1207.4684cs.DSstat.ML
收藏

快速柯西变换与快速稳健线性回归

The Fast Cauchy Transform and Faster Robust Linear Regression

Kenneth L. Clarkson, Petros Drineas, Malik Magdon-Ismail, Michael W. Mahoney, Xiangrui Meng, David P. Woodruff

摘要:

我们提供了快速算法的过约束$\ell_p$回归和相关问题:对于一个$n\times d$输入矩阵$A$和矢量$b\in\mathbb{R}^n$,在$O(nd\log n)$时间,我们减少了问题$\min_{x\in\mathbb{R}^d} \|Ax-b\|_p$到相同的问题与输入矩阵$\tilde A$的尺寸$s \times d$和相应的$\tilde b$的尺寸$s\times 1$。这里,$\tilde A$和$\tilde b$是问题的核心集,由$A$和$b$的采样和重新缩放的行组成; $s$独立于$n$和$d$中的多项式。我们的研究结果改善了以前最好的算法时,$n\gg d$,所有$p\in[1,\infty)$除了$p=2$。我们还提供了一套改进的结果,通过椭圆舍入找到良好的条件基地,说明运行时间和空调质量之间的权衡,包括一般$\ell_p$问题的一个通过空调算法。我们还提供了一个实证评估我们的算法$p=1$的实现,比较它们与相关算法。我们的实证结果表明,在渐近制度,理论是一个很好的指导这些算法的实际性能。我们的算法使用我们更快的构造$\ell_p$空间的良好条件基,对于$p=1$,我们称之为快速柯西变换的独立兴趣的快速子空间嵌入:矩阵$Π:\mathbb{R}^n\mapsto \mathbb{R}^{O(d\log d)}$上的分布,发现与$A$无关,近似保持$\ell_1$范数:也就是说,对于任意小常数$η>0$,对于所有$x$,$\|Ax\|_1 \approx \|ΠAx\|_1$,同时具有失真$O(d^{2+η})$,具有大概率;而且,$ΠA$可以在$O(nd\log d)$时间内计算。我们的快速柯西变换的基础技术包括快速约翰逊-林登施特劳斯变换,低相干矩阵,柯西随机变量的重新缩放。

英文摘要:

We provide fast algorithms for overconstrained $\ell_p$ regression and related problems: for an $n\times d$ input matrix $A$ and vector $b\in\mathbb{R}^n$, in $O(nd\log n)$ time we reduce the problem $\min_{x\in\mathbb{R}^d} \|Ax-b\|_p$ to the same problem with input matrix $\tilde A$ of dimension $s \times d$ and corresponding $\tilde b$ of dimension $s\times 1$. Here, $\tilde A$ and $\tilde b$ are a coreset for the problem, consisting of sampled and rescaled rows of $A$ and $b$; and $s$ is independent of $n$ and polynomial in $d$. Our results improve on the best previous algorithms when $n\gg d$, for all $p\in[1,\infty)$ except $p=2$. We also provide a suite of improved results for finding well-conditioned bases via ellipsoidal rounding, illustrating tradeoffs between running time and conditioning quality, including a one-pass conditioning algorithm for general $\ell_p$ problems. We also provide an empirical evaluation of implementations of our algorithms for $p=1$, comparing them with related algorithms. Our empirical results show that, in the asymptotic regime, the theory is a very good guide to the practical performance of these algorithms. Our algorithms use our faster constructions of well-conditioned bases for $\ell_p$ spaces and, for $p=1$, a fast subspace embedding of independent interest that we call the Fast Cauchy Transform: a distribution over matrices $Π:\mathbb{R}^n\mapsto \mathbb{R}^{O(d\log d)}$, found obliviously to $A$, that approximately preserves the $\ell_1$ norms: that is, with large probability, simultaneously for all $x$, $\|Ax\|_1 \approx \|ΠAx\|_1$, with distortion $O(d^{2+η})$, for an arbitrarily small constant $η>0$; and, moreover, $ΠA$ can be computed in $O(nd\log d)$ time. The techniques underlying our Fast Cauchy Transform include fast Johnson-Lindenstrauss transforms, low-coherence matrices, and rescaling by Cauchy random variables.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑