The primary goal in this thesis is to identify better syntactic constraint or bias, that is language independent but also efficiently exploitable during sentence processing. We focus on a particular syntactic construction called center-embedding, which is well studied in psycholinguistics and noted to cause particular difficulty for comprehension. Since people use language as a tool for communication, one expects such complex constructions to be avoided for communication efficiency. From a computational perspective, center-embedding is closely relevant to a left-corner parsing algorithm, which can capture the degree of center-embedding of a parse tree being constructed. This connection suggests left-corner methods can be a tool to exploit the universal syntactic constraint that people avoid generating center-embedded structures. We explore such utilities of center-embedding as well as left-corner methods extensively through several theoretical and empirical examinations.
Our primary task is unsupervised grammar induction. In this task, the input to the algorithm is a collection of sentences, from which the model tries to extract the salient patterns on them as a grammar. This is a particularly hard problem although we expect the universal constraint may help in improving the performance since it can effectively restrict the possible search space for the model. We build the model by extending the left-corner parsing algorithm for efficiently tabulating the search space except those involving center-embedding up to a specific degree. We examine the effectiveness of our approach on many treebanks, and demonstrate that often our constraint leads to better parsing performance. We thus conclude that left-corner methods are particularly useful for syntax-oriented systems, as it can exploit efficiently the inherent universal constraints in languages.
Transition-based Semantic Role Labeling with Pointer Networks
Universidade da Coruña（拉科鲁尼亚大学）
语义角色标注（Semantic role labeling，SRL）主要用于识别句子中的谓词-论元结构，在机器翻译和问答等自然语言处理任务中起着重要作用。实际上，所有可用的方法都不执行完整的SRL，因为它们依赖于预先确定的谓词，并且它们中的大多数遵循管道策略，使用特定的模型来执行一个或多个SRL子任务。此外，以前的方法有很强的依赖于语法信息，以实现最先进的性能，尽管语法树同样难以产生。这些简化和要求使得大多数SRL系统对于现实世界的应用是不切实际的。在这篇文章中，我们提出了第一个基于转换的SRL方法，该方法能够在一个从左到右的过程中完全处理输入句子，既不利用句法信息，也不诉诸于额外的模块。得益于我们基于指针网络的实现，完整的SRL可以在$O(n^2)$中准确高效地完成，在CoNLL-2009共享任务的大多数语言上实现了迄今为止的最佳性能。
Hierarchical Frequency Tagging Probe (HFTP): A Unified Approach to Investigate Syntactic Structure Representations in Large Language Models and the Human Brain
Peking University（北京大学）；Zhejiang University（浙江大学）；Beijing Language and Culture University（北京语言大学）；Beijing Institute for General Artificial Intelligence（北京通用人工智能研究院）
