结构化模型中快速推理的低秩约束
Low-Rank Constraints for Fast Inference in Structured Models
摘要：
结构化分布，即组合空间上的分布，通常用于从观察到的数据中学习潜在的概率表示。然而，缩放这些模型是由高的计算和内存复杂度相对于潜在的表示的大小的检查。常见的模型，如隐马尔可夫模型（HMM）和概率上下文无关文法（PCFG）需要的时间和空间的隐藏状态的数量分别为平方和立方。这项工作演示了一个简单的方法，以减少计算和内存的复杂性的一大类结构化模型。我们表明，通过将中央推理步骤视为矩阵向量乘积并使用低秩约束，我们可以通过秩来权衡模型表达能力和速度。神经参数化结构化模型的语言建模，复调音乐建模，无监督语法归纳和视频建模的实验表明，我们的方法在大的状态空间匹配标准模型的准确性，同时提供实际的加速。
英文摘要：
Structured distributions, i.e. distributions over combinatorial spaces, are commonly used to learn latent probabilistic representations from observed data. However, scaling these models is bottlenecked by the high computational and memory complexity with respect to the size of the latent representations. Common models such as Hidden Markov Models (HMMs) and Probabilistic Context-Free Grammars (PCFGs) require time and space quadratic and cubic in the number of hidden states respectively. This work demonstrates a simple approach to reduce the computational and memory complexity of a large class of structured models. We show that by viewing the central inference step as a matrix-vector product and using a low-rank constraint, we can trade off model expressivity and speed via the rank. Experiments with neural parameterized structured models for language modeling, polyphonic music modeling, unsupervised grammar induction, and video modeling show that our approach matches the accuracy of standard models at large state spaces while providing practical speedups.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
