用于压缩 Tucker 分解的不经意子空间嵌入
Oblivious subspace embeddings for compressed Tucker decompositions
AI总结:
针对 Tucker 分解缺乏随机嵌入理论的问题,本文证明沿各模态使用不经意 JL 嵌入的误差保证,并通过随机化 HOOI 在图像和 fMRI 数据上实现显著降本且误差几乎不增。
摘要:
张量文献中对随机嵌入(用于低失真降维的工具)的研究重点集中在 canonical polyadic(CP)张量分解上,这使得表达能力更强的 Tucker 分解的相应结果相对缺乏。本工作为沿每个模态应用不经意随机嵌入时的 Tucker 分解估计建立了通用的 Johnson-Lindenstrauss(JL)型保证。当这些嵌入取自 JL 最优族时,在与近期 CP 结果一致的嵌入维度限制下,分解可在 ε 相对误差内得到估计。我们实现了一种带随机嵌入的高阶正交迭代(HOOI)分解算法,以展示该方法的实际收益,以及其提升原本代价高昂的张量分析可及性的潜力。在中等规模的人脸图像和 fMRI 神经影像数据集上,实证结果表明,相对于传统 HOOI,可以实现大幅降维,而重建误差仅有极小增加(误差增大小于等于 5%;在沿每个模态降维 50% 的情况下,大型模型的计算时间降低 50%–60%)。尤其对于大型张量,我们的方法优于传统的高阶奇异值分解(HOSVD)以及近期提出的 TensorSketch 方法。
英文摘要:
Emphasis in the tensor literature on random embeddings (tools for low-distortion dimension reduction) for the canonical polyadic (CP) tensor decomposition has left analogous results for the more expressive Tucker decomposition comparatively lacking. This work establishes general Johnson-Lindenstrauss (JL) type guarantees for the estimation of Tucker decompositions when an oblivious random embedding is applied along each mode. When these embeddings are drawn from a JL-optimal family, the decomposition can be estimated within $\varepsilon$ relative error under restrictions on the embedding dimension that are in line with recent CP results. We implement a higher-order orthogonal iteration (HOOI) decomposition algorithm with random embeddings to demonstrate the practical benefits of this approach and its potential to improve the accessibility of otherwise prohibitive tensor analyses. On moderately large face image and fMRI neuroimaging datasets, empirical results show that substantial dimension reduction is possible with minimal increase in reconstruction error relative to traditional HOOI ($\leq$5% larger error, 50%-60% lower computation time for large models with 50% dimension reduction along each mode). Especially for large tensors, our method outperforms traditional higher-order singular value decomposition (HOSVD) and recently proposed TensorSketch methods.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。