使用复随机变量改进TensorSketch

中文摘要 AI 辅助

TensorSketch（由文献pham2013fast、kar2012random提出）为高维多项式核$\boldsymbol{x}^{\bigotimes p} \notin \boldsymbol{R}^{d^p}$提供高效的sketching算法。文献kar2012random采用稠密Johnson-Lindenstrauss（JL）型投影，计算复杂度为$O(pDd)$（其中$D$为sketch维度）；文献pham2013fast扩展了稀疏CountSketch算法，针对高维稀疏输入实现更快算法，运行时间为$O\big(p(\text{nnz}\boldsymbol{x} + D \text{log} D)\big)$。然而两种估计量的方差均随多项式次数$p$指数增长，缩放为$3^p/D$。近期文献pmlr-v206-wacker23a表明，对文献kar2012random的方法使用复值分布可将该依赖关系降至$2^p/D$，但该方法依赖稠密JL型投影，计算复杂度为$O(pDd)$，无法扩展至文献pham2013fast的算法。本研究提出文献pham2013fast的TensorSketch变体，达到与文献pmlr-v206-wacker23a相同的方差界，同时保留其输入稀疏性运行时间优势，通过合成数据集与真实世界数据集上的实验验证了结果。

英文摘要

\texttt{TensorSketch} by~\cite{pham2013fast,kar2012random} provides efficient sketching algorithms for high-dimensional polynomial kernels $\vec{x}^{\otimes p} \in \R^{d^p}$. \cite{kar2012random} uses dense Johnson-Lindenstrauss (JL)-type projections with computational cost $O(pDd)$, where $D$ denotes the sketch dimension, whereas~\cite{pham2013fast} extends the sparse \texttt{CountSketch}~\citep{count_sketch} algorithm, yielding a faster algorithm for high-dimensional sparse inputs with running time $O\big(p(\nnz{\vec{x}} + D \log D)\big)$. However, the variance of both estimators grows exponentially with the polynomial degree $p$, scaling as $3^{p}/D$. Recent work by~\cite{pmlr-v206-wacker23a} showed that using complex-valued distribution reduces this dependence to $2^{p}/D$ for the approach of~\cite{kar2012random}. However, their method relies on dense JL-type projections with computational cost $O(pDd)$ and does not extend to the algorithm of~\cite{pham2013fast}. In this work, we introduce a simple variant of \texttt{TensorSketch}~\citep{pham2013fast} that achieves the same variance bound as~\cite{pmlr-v206-wacker23a}, while retaining its advantage of the input-sparsity running time. We validate our results with supporting experiments on synthetic and real-world datasets.

发表机构

• IIT Hyderabad（印度理工学院海得拉巴分校）
• Bucknell University（巴克内尔大学）

机构由 AI 辅助整理，请以论文原文为准。