在语言模型避免过度概括时，区分统计优先与固化效应
Disentangling Statistical Preemption from Entrenchment in Language Models' Avoidance of Overgeneralization
浏览论文内容
中文总结 AI 辅助
该研究通过对语言模型开展受控实验，区分了语言模型避免过度概括时的优先效应与固化效应，发现其在动词特定层面无优先效应，仅存在微弱抽象优先效应，还揭示了模型对竞争结构的证据处理方式，为相关人类实验提供了新方向。
中文摘要 AI 辅助
学习者如何在缺乏明确负面证据的情况下避免诸如“Tom laughed me”这类过度概括？构式主义者提出了两种描述反对过度概括的间接负面证据的假说：优先效应（preemption，指学习者接触近义构式的情况，例如“she made him laugh”）与固化效应（entrenchment，指动词所有语法用法的接触情况，包括“He laughed”这类案例）。我们通过对在儿童-看护者对话上训练的语言模型（LMs）开展受控“养育实验”来区分这些假说，在此过程中我们系统地移除优先性证据与非优先性证据。我们发现，尽管语言模型会避免过度概括，但它们在动词特定层面并未表现出优先效应，反而呈现出微弱但非零的抽象优先效应证据。结合对语言模型训练动态的分析结果，我们发现，在动词特定条件下，语言模型将竞争结构视为间接正面证据，而非负面证据。就优先效应是人类避免过度概括更合理的路径而言，我们的结果表明神经网络学习者需要对间接负面证据具有敏感性，并为检验抽象优先效应提出了新的人类实验方向。
英文摘要
How do learners avoid overgeneralizations such as Tom laughed me without explicit negative evidence? Constructionists have posited two proposals that describe indirect negative evidence against overgeneralizations: preemption (which privileges exposure to near-synonymous construction---e.g., she made him laugh) vs. entrenchment (all exposures to a verb's grammatical usages, including cases like He laughed). We disentangle these hypotheses by running controlled rearing experiments on LMs trained on child-caregiver conversations, where we systematically remove preemptive vs. non-preemptive evidence. We find that while LMs avoid overgeneralizations, they do not show preemption at a verb-specific level, instead showing weak but non-zero evidence of abstract preemption. Combined with results from analyzing the LMs' training dynamics, we find that LMs treat competing structures as indirect positive---as opposed to negative---evidence in the verb-specific condition. Insofar as preemption is the more plausible route to avoiding overgeneralizations in humans, our results point the need for there to be sensitivities to indirect negative evidence in neural network learners, and suggest new human experiments to test abstract preemption.
发表机构
- University of Waterloo（滑铁卢大学）
- Vector Institute（向量研究所）
- University of Texas at Austin（德克萨斯大学奥斯汀分校）
机构由 AI 辅助整理，请以论文原文为准。
