发表机构
University of Washington（华盛顿大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究发现不同规模的语言模型可从贫乏输入中泛化出类人的范围同态名词短语修饰语顺序偏好，且该偏好无法用点互信息解释。
AI 中文摘要
语言习得领域的核心问题之一是，语言偏见是否能从作用于欠充分输入的通用学习机制中产生。人工语言学习（Artificial Language Learning，ALL）研究表明，人类学习者会可靠地从给定证据中进行泛化，包括倾向于范围同态的名词短语修饰语顺序。本研究探究语言模型（Language Models，LMs）在类似条件下是否表现出相同偏见。我们构建了受控学习环境：训练语料中移除了所有含多个修饰语的名词短语，消除了修饰语顺序的直接证据，之后在多修饰语句子上对模型进行评估。在三种模型规模下，我们发现模型在训练期间从未观察到范围同态顺序的情况下，仍始终偏好此类顺序，且这些偏好的强度随修饰语类型变化。为探究偏好来源，我们用点互信息（pointwise mutual information，PMI）分析名词-修饰语关联强度。尽管PMI反映了已知的修饰语顺序模式，但无法解释模型的顺序偏好。这些发现表明，语言模型可从贫乏输入中恢复类人的语言泛化能力，并为研究此类偏见的潜在机制提供了受控框架。
英文摘要
A central question in language acquisition is whether linguistic biases can emerge from general learning mechanisms operating over underdetermined input. Artificial Language Learning (ALL) studies have shown that human learners reliably generalize beyond the evidence provided, including by preferring scope-homomorphic noun phrase modifier orders. In this work, we investigate whether language models exhibit the same bias under similar conditions. We create a controlled learning environment in which models are trained on a corpus where all noun phrases containing multiple modifiers have been removed, eliminating direct evidence about modifier ordering, and are then evaluated on multiple modifier sentences. Across three model sizes, we find that they consistently prefer scope-homomorphic orders despite never observing them during training. These preferences vary in strength by modifier type. To investigate the source of these preferences, we examine noun-modifier association strength using pointwise mutual information (PMI). While PMI reflects known modifier-ordering patterns, it does not explain the models' ordering preferences. These findings demonstrate that LMs can recover human-like linguistic generalizations from impoverished input and provide a controlled framework for investigating the mechanisms underlying such biases.
