Kallini et al. (2024) 未将不可能语言与基于成分的语言进行比较
Kallini et al. (2024) do not compare impossible languages with constituency-based ones
AI总结:
本文指出Kallini et al. (2024)在验证LLMs归纳偏置的实验中存在混淆因素,导致其关于LLMs难以学习“不可能”语言的结论不成立,并提出了改进比较方法的建议。
摘要:
语言学理论的一个核心目标是找到对“可能的人类语言”这一概念的精确刻画,其形式为一种计算装置,能够描述所有且仅能描述可被正常发育的人类儿童习得的语言。近期大型语言模型(LLMs)在NLP应用中的成功,可以说提高了LLMs可能成为满足这一目标的计算装置的可能性。只有当LLMs不仅成功学习了人类语言,而且在学习“不可能的”人类语言时表现吃力,这种情况才会成立。Kallini et al. (2024; "Mission: Impossible Language Models", Proc. ACL) 进行了旨在测试这一点的实验,通过在多种合成语言上训练GPT-2,发现它对某些语言的学习比其他语言更成功。他们将这些不对称性作为支持LLMs的归纳偏置与人类语言中“可能”的概念相一致这一观点的证据,但最重要的比较存在一个混淆因素,使得该结论无法成立。在本文中,我解释了该混淆因素,并提出了一些推进方向,以构建一个能恰当测试潜在问题的比较。
英文摘要:
A central goal of linguistic theory is to find a precise characterization of the notion "possible human language", in the form of a computational device that is capable of describing all and only the languages that can be acquired by a typically developing human child. The success of recent large language models (LLMs) in NLP applications arguably raises the possibility that LLMs might be computational devices that meet this goal. This would only be the case if, in addition to succeeding in learning human languages, LLMs struggle to learn "impossible" human languages. Kallini et al. (2024; "Mission: Impossible Language Models", Proc. ACL) conducted experiments aiming to test this by training GPT-2 on a variety of synthetic languages, and found that it learns some more successfully than others. They present these asymmetries as support for the idea that LLMs' inductive biases align with what is regarded as "possible" for human languages, but the most significant comparison has a confound that makes this conclusion unwarranted. In this paper I explain the confound and suggest some ways forward towards constructing a comparison that appropriately tests the underlying issue.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。