Techniques in which words are represented as vectors have proved useful in many applications in computational linguistics, however there is currently no general semantic formalism for representing meaning in terms of vectors. We present a framework for natural language semantics in which words, phrases and sentences are all represented as vectors, based on a theoretical analysis which assumes that meaning is determined by context.
In the theoretical analysis, we define a corpus model as a mathematical abstraction of a text corpus. The meaning of a string of words is assumed to be a vector representing the contexts in which it occurs in the corpus model. Based on this assumption, we can show that the vector representations of words can be considered as elements of an algebra over a field. We note that in applications of vector spaces to representing meanings of words there is an underlying lattice structure; we interpret the partial ordering of the lattice as describing entailment between meanings. We also define the context-theoretic probability of a string, and, based on this and the lattice structure, a degree of entailment between strings.
We relate the framework to existing methods of composing vector-based representations of meaning, and show that our approach generalises many of these, including vector addition, component-wise multiplication, and the tensor product.
On the Compositionality Prediction of Noun Phrases using Poincaré Embeddings
Indian Institute of Technology Kharagpur（印度理工学院卡拉格普尔分校）；National Research University Higher School of Economics（国立研究型大学高等经济学院）；Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究院）；Universität Hamburg（汉堡大学）
Bridging Compositional and Distributional Semantics: A Survey on Latent Semantic Geometry via AutoEncoder
Department of Computer Science, University of Manchester（曼彻斯特大学计算机科学系）；Idiap Research Institute（Idiap研究 institute）；Cancer Biomarker Centre, CRUK Manchester Institute（CRUK曼彻斯特癌症生物标志物中心）
加法合成（Foltz et al，1998; Landauer and Dumais，1997; Mitchell and Lapata，2010）是一种广泛使用的计算短语含义的方法，它取组成词的向量表示的平均值。在这篇文章中，我们证明了一个上界的偏见的加法组合，这是第一次从机器学习的角度对组合框架的理论分析。的结合是书面的搭配强度，我们证明，更专门的两个连续的单词往往会一起出现，更准确的可以保证其添加剂组成的自然短语向量的近似。我们的证明依赖于经验验证的自然语言数据的属性，并且可以从理论上推导出数据是从分层Pitman-Yor过程生成的假设。该理论认为加法合成是计算短语含义的合理操作，并提出了改进加法合成的方法，包括：通过满足特定条件的函数转换分布式词向量的条目，构造一种新型的向量表示使加法合成对词序敏感，以及利用奇异值分解训练词向量。
Distributional Sentence Entailment Using Density Matrices
Queen Mary University of London（伦敦玛丽女王大学）；University of Oxford（牛津大学）
Coecke et al.（2010）的范畴组合分布模型提出了一种将形式的类型逻辑模型的语法组合与基于语料库的分布语义的经验词表示相结合的方法。本文通过扩展模型来捕获蕴涵关系，从而为该项目做出了贡献。这是通过将词的表示从意义空间中的点扩展到密度算子来实现的，密度算子是空间子空间上的概率分布。定义了相似性的对称度量和蕴涵的非对称度量，其中词汇蕴涵使用冯·诺依曼熵（Kullback-Leibler发散的量子变体）来测量。词汇蕴涵与词表征的组合图相结合，提供了一种在句子层次上获取蕴涵关系的方法。真值理论和语料库为基础的例子。
