arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2203.01599cs.LGcs.CGcs.DSstat.ML
收藏

随机Hadamard变换的一致逼近及其应用

Uniform Approximations for Randomized Hadamard Transforms with Applications

UC Berkeley(加州大学伯克利分校)

Yeshwanth Cherapanamjeri, Jelani Nelson

摘要:

随机阿达玛变换(RHT)已经成为在计算机科学和机器学习的一系列领域中使用密集非结构化随机矩阵的计算上有效的替代方案。对于降维和压缩感知等应用,基于RHT的方法的理论保证与使用具有i.i.d.的稠密随机矩阵的方法相当。条目然而,几个这样的应用程序是在低维制度,从矩阵中采样的行数是相当小的。先验参数不适用于在核近似等机器学习应用中经常发现的高维区域。给定一个系综的RHTs与高斯对角线,$\{M^i\}_{i = 1}^m$,和任何$1$ -Lipschitz函数,$f: \mathbb{R} \to \mathbb{R}$,我们证明了平均$f$的条目$\{M^i v\}_{i = 1}^m$收敛到其期望均匀超过$\| v \| \leq 1$的速度可比使用真正的高斯矩阵。我们使用我们的不等式,然后得到改进的保证,在高维制度的两个应用程序:1)核近似和2)距离估计。对于核近似,我们证明了第一个\emph{uniform}近似保证随机功能,通过RHTs贷款理论证明其经验的成功,而距离估计,我们的收敛结果意味着数据结构与改进的运行时保证作者以前的工作。我们相信我们的一般不等式可能会在其他应用中找到用途。

英文摘要:

Randomized Hadamard Transforms (RHTs) have emerged as a computationally efficient alternative to the use of dense unstructured random matrices across a range of domains in computer science and machine learning. For several applications such as dimensionality reduction and compressed sensing, the theoretical guarantees for methods based on RHTs are comparable to approaches using dense random matrices with i.i.d.\ entries. However, several such applications are in the low-dimensional regime where the number of rows sampled from the matrix is rather small. Prior arguments are not applicable to the high-dimensional regime often found in machine learning applications like kernel approximation. Given an ensemble of RHTs with Gaussian diagonals, $\{M^i\}_{i = 1}^m$, and any $1$-Lipschitz function, $f: \mathbb{R} \to \mathbb{R}$, we prove that the average of $f$ over the entries of $\{M^i v\}_{i = 1}^m$ converges to its expectation uniformly over $\| v \| \leq 1$ at a rate comparable to that obtained from using truly Gaussian matrices. We use our inequality to then derive improved guarantees for two applications in the high-dimensional regime: 1) kernel approximation and 2) distance estimation. For kernel approximation, we prove the first \emph{uniform} approximation guarantees for random features constructed through RHTs lending theoretical justification to their empirical success while for distance estimation, our convergence result implies data structures with improved runtime guarantees over previous work by the authors. We believe our general inequality is likely to find use in other applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑