SGD_Tucker：一种新的并行稀疏Tucker分解随机优化策略
SGD_Tucker: A Novel Stochastic Optimization Strategy for Parallel Sparse Tucker Decomposition
摘要：
稀疏塔克分解（STD）算法学习核心张量和一组因子矩阵以获得用于\underline{H}高- \underline{O}序列、\underline{H}高- \underline{D}维数和\underline{S}解析\underline{T}传感器（HOHDST）的最佳低秩表示特征。然而，现有的STD算法面临着中间变量爆炸的问题，这是由于这些变量的形成，即，矩阵的Khatri-Rao乘积、Kronecker乘积和矩阵-矩阵乘法，跟随稀疏张量中的全部元素。上述问题阻碍了高效计算和大数据平台的深度融合。为克服这一瓶颈，提出了一种新的随机优化策略（SGD $\_$ Tucker），该策略能自动将高维中间变量分解为小批量的中间矩阵.具体来说，SGD $\_$ Tucker只跟随随机选择的小样本而不是整个元素，同时保持整体精度和收敛速度。在实践中，SGD $\_$ Tucker具有两个超越现有技术的显著进步：首先，SGD $\_$ Tucker可以在分布式设置中削减核心张量的通信开销。其次，SGD $\_$ Tucker的低数据依赖性使得能够实现细粒度并行化，这使得SGD $\_$ Tucker在相同精度的情况下获得更低的计算开销。实验结果表明，SGD $\_$ Tucker运行速度比现有技术至少快2个$X$。
英文摘要：
Sparse Tucker Decomposition (STD) algorithms learn a core tensor and a group of factor matrices to obtain an optimal low-rank representation feature for the \underline{H}igh-\underline{O}rder, \underline{H}igh-\underline{D}imension, and \underline{S}parse \underline{T}ensor (HOHDST). However, existing STD algorithms face the problem of intermediate variables explosion which results from the fact that the formation of those variables, i.e., matrices Khatri-Rao product, Kronecker product, and matrix-matrix multiplication, follows the whole elements in sparse tensor. The above problems prevent deep fusion of efficient computation and big data platforms. To overcome the bottleneck, a novel stochastic optimization strategy (SGD$\_$Tucker) is proposed for STD which can automatically divide the high-dimension intermediate variables into small batches of intermediate matrices. Specifically, SGD$\_$Tucker only follows the randomly selected small samples rather than the whole elements, while maintaining the overall accuracy and convergence rate. In practice, SGD$\_$Tucker features the two distinct advancements over the state of the art. First, SGD$\_$Tucker can prune the communication overhead for the core tensor in distributed settings. Second, the low data-dependence of SGD$\_$Tucker enables fine-grained parallelization, which makes SGD$\_$Tucker obtaining lower computational overheads with the same accuracy. Experimental results show that SGD$\_$Tucker runs at least 2$X$ faster than the state of the art.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
