发表机构
Mohamed bin Zayed University of Artificial Intelligence; Tohoku University（穆罕默德·本·扎耶德人工智能大学; 东北大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文通过基于栈的语言模型在人工语言上的实验，发现有限工作记忆有助于泛化交叉序列依赖，为语言类型学共性提供了归纳偏差的可能基础。
AI 中文摘要
在数千种已证实的自然语言中，某些语言属性（如主宾动词序）比其他属性更为普遍。这种类型学共性通常归因于学习偏差。近期使用语言模型的计算模拟促进了这一理论的探索。在本文中，我们扩展了现有关于语言模型学习偏差与类型学共性之间关系的数据和模型两方面的分析，重点关注：（i）交叉序列依赖，即已证实句法复杂性的上限，以及（ii）基于栈的语言模型，其可能促进层次模式的学习。我们首先评估了基于栈的语言模型在各种人工语言上对交叉序列依赖的泛化能力，并确认它们在处理此类结构时存在困难。然而，具有有限工作记忆的基于栈的语言模型泛化能力更好，这为这种归纳偏差以及某些词序配置的类型学共性提供了可能的基础。
英文摘要
Some properties of languages, e.g., subject-object-verb (SOV) word order, are more prevalent than others among the thousands of attested natural languages (NLs). Such typological commonality is often attributed to learning biases. Computational simulations, recently with language models (LMs), have facilitated the exploration of this theory. In this paper, we extend existing analyses of the relationship between LMs' learning biases and typological commonality on both data and model sides, focusing on: (i) cross-serial dependencies, the upper limit of attested syntactic complexity, and (ii) stack-based LMs (SLMs), potentially facilitating learning of hierarchical patterns. We first evaluate generalization of SLMs on cross-serial dependencies across diverse artificial languages and confirm that they struggle with such constructions. However, SLMs with limited working memory generalize better suggesting a possible basis for such inductive bias and thus the typological commonality of some word order configurations.
CommentsEMNLP 2026 Main Conference
