arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1708.01525cs.CLcond-mat.str-elphysics.hist-phquant-ph
收藏

语言设计作为信息重整

Language Design as Information Renormalization

Universitat Autònoma de Barcelona(巴塞罗那自治大学);Donostia International Physics Center(多诺斯蒂亚国际物理中心);Ikerbasque Foundation for Science(伊克尔巴斯克科学基金会);Multiverse Computing(多元宇宙计算公司)

Angel J. Gallego, Roman Orus

摘要:

在这里,我们从物理学的角度考虑语法中的一些众所周知的事实,使我们能够在两个领域之间建立等价关系,并产生许多后果。我们主要观察到N. Chomsky在1995年提出的,可以解释为物理信息的粗粒化。因此,语言学中的MERGE需要物理学中的信息重整,根据不同的时间尺度。我们用语言模型使这一点在数学上形式化。在此设置中,MERGE相当于实现粗粒度的概率张量,类似于概率上下文无关文法。有意义句子的概率向量由随机张量网络(TN)给出,这些网络由对角张量构建,并且大多是无环的,例如树张量网络和矩阵乘积状态,因此在计算上非常有效。我们表明,这意味着多项式衰减(长程)相关性实验观察到的语言,也提供了参数,有利于某些类型的神经网络的语言处理。此外,我们展示了如何获得这样的语言模型从量子态,可以有效地准备在量子计算机上,并使用它来找到一个句子中的单词的概率分布的困惑的界限。我们的研究结果的影响进行了讨论,在几个范围。

英文摘要:

Here we consider some well-known facts in syntax from a physics perspective, allowing us to establish equivalences between both fields with many consequences. Mainly, we observe that the operation MERGE, put forward by N. Chomsky in 1995, can be interpreted as a physical information coarse-graining. Thus, MERGE in linguistics entails information renormalization in physics, according to different time scales. We make this point mathematically formal in terms of language models. In this setting, MERGE amounts to a probability tensor implementing a coarse-graining, akin to a probabilistic context-free grammar. The probability vectors of meaningful sentences are given by stochastic tensor networks (TN) built from diagonal tensors and which are mostly loop-free, such as Tree Tensor Networks and Matrix Product States, thus being computationally very efficient to manipulate. We show that this implies the polynomially-decaying (long-range) correlations experimentally observed in language, and also provides arguments in favour of certain types of neural networks for language processing. Moreover, we show how to obtain such language models from quantum states that can be efficiently prepared on a quantum computer, and use this to find bounds on the perplexity of the probability distribution of words in a sentence. Implications of our results are discussed across several ambits.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑