注意张量积学习
Attentive Tensor Product Learning
摘要：
本文提出了一种新的架构-注意力张量积学习（ATPL）-来表示深度学习模型中的语法结构。ATPL是一种新的架构，通过利用张量乘积表示（TPR）来弥合这一差距，TPR是认知科学中开发的一种结构化神经符号模型，旨在将深度学习与显式语言结构和规则集成在一起。ATPL的主要思想是：1）通过基于TPR的深度神经网络对单词的角色解除绑定向量进行无监督学习; 2）采用注意力模块来计算TPR;以及3）将TPR与典型的深度学习架构集成，包括长短期记忆（LSTM）和前馈神经网络（FFNN）。我们的方法的新颖之处在于它能够提取一个句子的语法结构，通过使用角色解除约束的向量，这是在一个无监督的方式获得。该ATPL方法被应用于1）图像描述，2）词性（POS）标记，和3）句子的选区解析。实验结果证明了该方法的有效性。
英文摘要：
This paper proposes a new architecture - Attentive Tensor Product Learning (ATPL) - to represent grammatical structures in deep learning models. ATPL is a new architecture to bridge this gap by exploiting Tensor Product Representations (TPR), a structured neural-symbolic model developed in cognitive science, aiming to integrate deep learning with explicit language structures and rules. The key ideas of ATPL are: 1) unsupervised learning of role-unbinding vectors of words via TPR-based deep neural network; 2) employing attention modules to compute TPR; and 3) integration of TPR with typical deep learning architectures including Long Short-Term Memory (LSTM) and Feedforward Neural Network (FFNN). The novelty of our approach lies in its ability to extract the grammatical structure of a sentence by using role-unbinding vectors, which are obtained in an unsupervised manner. This ATPL approach is applied to 1) image captioning, 2) part of speech (POS) tagging, and 3) constituency parsing of a sentence. Experimental results demonstrate the effectiveness of the proposed approach.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
