The use of null hypotheses (in a statistical sense) is common in hard sciences but not in theoretical linguistics. Here the null hypothesis that the low frequency of syntactic dependency crossings is expected by an arbitrary ordering of words is rejected. It is shown that this would require star dependency structures, which are both unrealistic and too restrictive. The hypothesis of the limited resources of the human brain is revisited. Stronger null hypotheses taking into account actual dependency lengths for the likelihood of crossings are presented. Those hypotheses suggests that crossings are likely to reduce when dependencies are shortened. A hypothesis based on pressure to reduce dependency lengths is more parsimonious than a principle of minimization of crossings or a grammatical ban that is totally dissociated from the general and non-linguistic principle of economy.
The Return of Lexical Dependencies: Neural Lexicalized PCFGs
Language Technologies Institute, Carnegie Mellon University(卡内基梅隆大学语言技术研究所)
在本文中,我们证明了$\textit{context free grammar (CFG) based methods for grammar induction benefit from modeling lexical dependencies}$。这与当前最流行的语法归纳方法形成对比,该方法专注于发现$\textit{either}$成分$\textit{or}$依赖性。以前的方法来结婚这两个不同的句法形式主义(例如词汇化PCFG)一直困扰着稀疏,使他们不适合无监督语法归纳。然而,在这项工作中,我们提出了新的词汇化PCFG的神经模型,使我们能够克服稀疏性问题,并有效地在一个单一的模型内诱导成分和依赖关系。实验表明,这种统一的框架结果在两个表示比单独建模时实现的形式主义更强的结果。代码可在https://github.com/neulab/neural-lpcfg上获得。
On the relation between dependency distance, crossing dependencies, and parsing. Comment on "Dependency distance: a new perspective on syntactic patterns in natural languages" by Haitao Liu et al
Universidade da Coruña(拉科鲁尼亚大学)
Liu et al.(2017)全面介绍了人类语言中依赖距离的研究。虽然这篇文章是关于这个复杂主题的非常丰富和有用的报告,但在这里我将扩展计算语言学(特别是自然语言处理)研究可以告知DDM研究的一些具体问题,反之亦然。这些方面在Liu等人的文章或其他地方没有进行太多的探索,可能是由于两个研究团体之间的重叠很少,但它们可能会为提高我们对人类语言进化的理解提供有趣的见解,大脑处理和理解语言的机制,以及构建有效的计算机系统来实现这一目标。
Reasoning over Grammar: Can Synthetic Linguistic Reasoning Traces Enhance Low-Resource Machine Translation?
ELLIS Institute Finland(芬兰ELLIS研究所);University of Turku(图尔库大学);Center for Information and Language Processing, LMU Munich(慕尼黑大学信息与语言处理中心);Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心)
Structural Supervision Improves Learning of Non-Local Grammatical Dependencies
Harvard University(哈佛大学);MIT(麻省理工学院);UC Irvine(加州大学欧文分校);IBM Research(IBM研究院);MIT-IBM Watson AI Lab(麻省理工学院-IBM沃森人工智能实验室)
在大型语料库上训练的最先进的LSTM语言模型以令人印象深刻的细节学习顺序偶然性,并已被证明获得了一些非局部语法依赖,并取得了一些成功。在这里,我们调查是否监督与层次结构增强学习的一系列语法依赖关系,以前只针对主谓一致的问题。使用来自心理语言学的受控实验方法,我们比较了基于单词的LSTM模型与两种表示层次结构并将其部署在从左到右处理中的模型的性能:递归神经网络语法(RNNG)(Dyer et al.,2016)和Chariak等人的Parsing-as-Question-Modeling配置的增量版本,(2016年)。模型在不同的配置范围内进行了测试,两类非本地语法依赖英语-负极性许可和填料-差距的依赖。使用相同的训练数据跨模型,我们发现结构监督模型优于LSTM,RNNG在两种类型的语法依赖关系上都表现出最佳结果,甚至学习了许多关于填充-间隙依赖关系的岛屿约束。因此,结构监督在获取关于非局部语法依赖性的类人概括方面提供了优于神经语言模型的纯基于字符串的训练的数据效率优势。
Shanghai Engineering Research Center of Intelligent Vision and Imaging(上海智能视觉与成像工程研究中心);ShanghaiTech University(上海科技大学);Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences(中国科学院上海微系统与信息技术研究所);University of Edinburgh(爱丁堡大学);University of Chinese Academy of Sciences(中国科学院大学)
Tree-Structured Neural Machine for Linguistics-Aware Sentence Generation
University of Chinese Academy of Sciences(中国科学院大学);Department of Computer Science, University of California Santa Barbara(加州大学圣塔芭芭拉分校计算机科学系);WeChat Search Application Department, Tencent(腾讯微信搜索应用部)
Local and non-local dependency learning and emergence of rule-like representations in speech data by Deep Convolutional Generative Adversarial Networks