伪装的样例：纯样例模型模仿优先抽象学习

中文摘要 AI 辅助

特定于项目的独特知识与抽象类级泛化知识究竟谁先被学习，是语言学习的核心问题，样例理论与基于抽象的理论对此作出相反预测。近期研究声称，至少对于大型语言模型而言，抽象知识优先被学习。我们证明这些方法存在不足：不含抽象表征的纯记忆模型，依据相同标准，可表现为优先学习项目特定知识或类级知识，具体取决于其对单个观测的敏感度，且该转变点由输入的分布特性决定。我们进一步提出，对于分布式表征而言，项目特定知识与抽象知识的区分可能定义不明确，因为词语的类级属性与其项目特定属性无法分离。

英文摘要

Whether idiosyncratic, item-specific knowledge is learned before abstract class-level generalizations, or vice versa, is a central question in language learning, with exemplar and abstraction-based theories making opposite predictions. Recent methods have claimed to show that, at least for large language models, abstract knowledge is learned first. We show that these methods fall short: pure memorizer models with no abstract representations can appear, by the same criteria, to learn either item-specific or class-level knowledge first, depending on their sensitivity to individual observations, with the transition point governed by the distributional properties of the input. We further argue that the distinction between item-specific and abstract knowledge may be ill-defined for distributed representations, as a word's class-level properties may not be separable from its item-specific properties.

发表机构

• University of Oregon（俄勒冈大学）
• Vail Systems, Inc.（韦尔系统公司）

机构由 AI 辅助整理，请以论文原文为准。