arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1808.09891cs.CL
收藏

量子多体波函数语言建模方法

A Quantum Many-body Wave Function Inspired Language Modeling Approach

Tianjin University(天津大学);University of Padova(帕多瓦大学);Beijing Institute of Technology(北京理工大学)

Peng Zhang, Zhan Su, Lipeng Zhang, Benyou Wang, Dawei Song

摘要:

最近提出的量子语言模型(QLM)旨在通过应用量子概率理论来建模术语依赖的原则性方法。更有效的QLM的最新发展采用了词嵌入作为一种全局依赖信息,并将量子启发的想法集成到神经网络架构中。虽然这些量子激发的LM理论上更通用,实际上也更有效,但它们有两个主要局限性。首先,他们没有考虑到具有多重含义的单词之间的相互作用,这在理解自然语言文本中是常见的和重要的。其次,量子LM与神经网络的集成主要是为了有效地训练参数,但缺乏理论基础来解释这种集成。为了解决这两个问题,在本文中,我们提出了量子多体波函数(QMWF)启发的语言建模方法。QMWF启发的LM可以采用张量积来建模词之间的上述交互。这也使我们能够揭示在QMWF语言建模中使用卷积神经网络(CNN)的内在必要性。此外,我们的方法提供了一个简单的算法来表示和匹配文本/句子对。在3个典型的问答(QA)数据集上,与量子LM算法和几种基于CNN的方法进行了比较,系统的评价表明了QMWF-LM算法的有效性.

英文摘要:

The recently proposed quantum language model (QLM) aimed at a principled approach to modeling term dependency by applying the quantum probability theory. The latest development for a more effective QLM has adopted word embeddings as a kind of global dependency information and integrated the quantum-inspired idea in a neural network architecture. While these quantum-inspired LMs are theoretically more general and also practically effective, they have two major limitations. First, they have not taken into account the interaction among words with multiple meanings, which is common and important in understanding natural language text. Second, the integration of the quantum-inspired LM with the neural network was mainly for effective training of parameters, yet lacking a theoretical foundation accounting for such integration. To address these two issues, in this paper, we propose a Quantum Many-body Wave Function (QMWF) inspired language modeling approach. The QMWF inspired LM can adopt the tensor product to model the aforesaid interaction among words. It also enables us to reveal the inherent necessity of using Convolutional Neural Network (CNN) in QMWF language modeling. Furthermore, our approach delivers a simple algorithm to represent and match text/sentence pairs. Systematic evaluation shows the effectiveness of the proposed QMWF-LM algorithm, in comparison with the state of the art quantum-inspired LMs and a couple of CNN-based methods, on three typical Question Answering (QA) datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑