词汇如何塑造语言模型中词序可学习性的跨语言变异
Vocabulary shapes cross-lingual variation of word-order learnability in language models
AI总结：
研究通过预训练变换器语言模型探讨不同词序语言的可学习性差异，发现词汇结构是影响词序可学习性的关键因素。
摘要：
为什么有些语言如捷克语允许自由词序，而其他语言如英语不允许？我们通过在多种合成词序变体上预训练变换器语言模型来解答这个问题。我们观察到，更大的词序不规则性会一致提高模型的意外程度，表明学习性降低。句子反转对学习性影响较弱。粗略区分自由词序语言（如捷克语和芬兰语）和固定词序语言（如英语和法语）无法解释跨语言变异。相反，词和子词词汇的结构强烈预测模型的意外程度。总体而言，词汇结构成为影响跨语言词序可学习性的关键因素。
英文摘要：
Why do some languages like Czech permit free word order, while others like English do not? We address this question by pretraining transformer language models on a spectrum of synthetic word-order variants of natural languages. We observe that greater word-order irregularity consistently raises model surprisal, indicating reduced learnability. Sentence reversal, however, affects learnability only weakly. A coarse distinction of free- (e.g., Czech and Finnish) and fixed-word-order languages (e.g., English and French) does not explain cross-lingual variation. Instead, the structure of the word and subword vocabulary strongly predicts the model surprisal. Overall, vocabulary structure emerges as a key driver of computational word-order learnability across languages.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
