发表机构
IBM; UMass Amherst（IBM; 麻省大学阿默斯特分校）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究针对Khatri-Rao无感知子空间嵌入，填补了现有理论差距，证明任意固定阶数的Khatri-Rao草图矩阵将$k$维子空间嵌入到$(1\pm \epsilon)$误差范围内，所需草图维度为$\tilde O(k/\epsilon^2)$。
AI 中文摘要
我们研究具有Khatri-Rao结构的随机草图矩阵。具体而言，我们考虑随机矩阵$A_i \in \mathbb R^{n_i \times m}$的Khatri-Rao乘积（即逐列张量积）$A_1\odot\cdots\odot A_d \in \mathbb R^{(n_1 \cdots n_d) \times m}$，其中$A_i$的列是各向同性、独立且次高斯的（例如高斯矩阵）。当输入数据具有张量结构，可与$A_1\odot\cdots\odot A_d$快速相乘时，Khatri-Rao草图矩阵被广泛应用于线性代数计算与数据分析的随机算法中。然而，现有理论无法完全解释其实际表现。特别是，尽管受到大量关注，我们对Khatri-Rao矩阵的重要无感知子空间嵌入性质的最优界，仍落后于标准非结构化矩阵可达到的水平。对于将$k$维子空间嵌入到$(1\pm \epsilon)$误差范围内，Bujanović等人在$d=2$的特殊情况下证明草图维度$m = O(k^{3/2}/\epsilon^2)$已足够，其对$k$的依赖弱于已知非结构化次高斯草图矩阵的紧界$O(k/\epsilon^2)$。本研究填补了这一差距，证明对于任意固定阶数$d$的Khatri-Rao草图矩阵，子空间嵌入所需的$m = \tilde O(k/\epsilon^2)$即可满足要求。我们的证明十分简洁，仅利用了Khatri-Rao草图分布的两个基本性质：1）$A_1\odot\cdots\odot A_d \in \mathbb R^{(n_1 \cdots n_d) \times m}$的列是独立且各向同性的；2）$A_1\odot\cdots\odot A_d \in \mathbb R^{(n_1 \cdots n_d) \times m}$的每一列均满足弱Johnson-Lindenstrauss型矩性质。
英文摘要
We study random sketching matrices with Khatri-Rao structure. In particular, we consider the Khatri-Rao product (i.e., column-wise tensor product) $A_1\odot\cdots\odot A_d \in \mathbb R^{(n_1 \cdots n_d) \times m}$ of random matrices $A_i \in \mathbb R^{n_i \times m}$ whose columns are isotropic, independent and sub-Gaussian (e.g., Gaussian matrices). Khatri-Rao sketching matrices are widely applied in randomized algorithms for linear algebraic computation and data analysis, when the input data has tensor structure that allows for fast multiplication with $A_1\odot\cdots\odot A_d$. However, existing theory is not able to fully explain their performance in practice. In particular, despite significant attention, our best bounds for the important \emph{oblivious subspace embedding} property with Khatri-Rao matrices lag behind what is achievable with standard unstructured matrices. For embedding a $k$-dimensional subspace to $(1\pm ε)$ error, Bujanović et al. \cite{bujanovic2025subspace} prove that sketching dimension $m = O(k^{3/2}/ε^2)$ suffices in the special case of $d = 2$. Their dependence on $k$ is weaker than the tight bound of $O(k/ε^2)$ known for unstructured sub-Gaussian sketching matrices. In this work, we close this gap, showing that $m = \tilde O(k/ε^2)$ suffices for subspace embedding with a Khatri-Rao sketching matrix with any fixed order $d$. Our proof is simple, leveraging just two basic properties of the Khatri-Rao sketching distribution: 1) the columns of $A_1\odot\cdots\odot A_d \in \mathbb R^{(n_1 \cdots n_d) \times m}$ are independent and isotropic, and 2) each column of $A_1\odot\cdots\odot A_d \in \mathbb R^{(n_1 \cdots n_d) \times m}$ satisfies a weak Johnson-Lindenstrauss type moment property.
