LLMs并非随机鹦鹉：来自类人工语言任务的语义中介抽象证据
LLMs are not stochastic parrots: Evidence for meaning-mediated abstraction from conlang-like tasks
浏览论文内容
中文总结 AI 辅助
通过类人工语言任务，证明LLM能基于提示中的约束进行语义中介的抽象，而非仅依赖表面统计模式，从而反驳强随机鹦鹉假说。
中文摘要 AI 辅助
随机鹦鹉假说的强版本声称，尽管大型语言模型（LLMs）可能超越死记硬背，但它们无法超越统计模式匹配进入抽象或推理，尽管生成看似流畅的文本，但在本体论上仍接近模式重用的下限。我们使用类人工语言任务来检验这一假设。多个LLM仅获得虚构语言的自然语言描述，这些描述通过组合统计上不常见且未经证实的特征，颠覆了训练数据中突出的表面模式。关键的是，没有给出任何示例输出。我们认为，如果模型表现出遵循规则的行为，它们不可能仅依赖表面的统计模式；这些模式往往与正确输出相悖。相反，成功的表现需要对提示中指定的约束进行表征。在三个互补的任务族中，模型系统性地朝着意义预测的方向移动：它们区分提示暴露与指示使用，响应新约束改变语义关系，有时对复杂的翻译答案键产生精确匹配。尽管不同模型的表现有所差异，但这些结果为LLM中的语义中介抽象提供了证据，并反驳了强随机鹦鹉假说。我们的工作表明，在适当的架构和上下文约束下，统计学习可以产生语义中介的抽象，尽管生成仍受到表面合理性的强烈限制。我们讨论了这些发现对模型开发以及理解日益抽象的表示如何从合理文本生成目标中涌现的意义。
英文摘要
The strong version of the stochastic parrot argument claims that, although large language models (LLMs) may exceed rote regurgitation, they cannot move beyond statistical pattern matching into abstraction or reasoning, remaining ontologically near the lower bound of pattern reuse despite producing alluringly fluent text. We test this hypothesis using conlang-like tasks. Several LLMs are given only natural-language descriptions of fictional languages that subvert prominent superficial patterns in training data by combining statistically uncommon and unattested features. Crucially, no example outputs are given. We argue that if the models exhibit rule-following behaviour, they cannot be relying solely on superficial statistical patterns; such patterns often work against the correct output. Instead, successful performance requires representations of the constraints specified in the prompt. Across three complementary task families, models systematically move in the meaning-predicted direction: they distinguish prompt exposure from instructed use, alter semantic relationships in response to novel constraints, and sometimes produce exact matches to complex translation answer keys. Although performance varies across the spectrum of models used, these results provide evidence for meaning-mediated abstraction in LLMs and refute the strong stochastic parrot hypothesis. Our work shows that, under appropriate architectural and contextual constraints, statistical learning can produce meaning-mediated abstractions, although generation remains strongly constrained by superficial plausibility. We discuss implications for model development and for understanding how increasingly abstract representations may emerge from plausible-text-generation objectives.
发表机构
- University of Vermont（佛蒙特大学）
机构由 AI 辅助整理，请以论文原文为准。
