从巧克力兔子到巧克力鳄鱼:语言模型理解名词复合词吗?
From chocolate bunny to chocolate crocodile: Do Language Models Understand Noun Compounds?
AI总结:
本文研究语言模型对名词复合词的理解能力,发现 GPT-3 在标准释义任务上几乎完美,在需要创造力和常识的新颖复合词概念化任务上优于人类,并分析了其推理与复述训练数据的程度。
摘要:
名词复合词释义任务旨在将名词复合词(如 chocolate bunny)表达为自由文本释义,以明确其构成名词之间的关系(如 bunny-shaped chocolate)。我们对标准任务(Hendrickx et al., 2013)的数据和评估设置提出了修改,并表明 GPT-3 几乎完美地解决了该任务。随后,我们研究了名词复合词概念化任务,即对新颖或罕见的名词复合词进行释义。例如,chocolate crocodile 是指鳄鱼形状的巧克力。该任务需要创造力、常识以及对相似概念知识进行泛化的能力。尽管 GPT-3 的表现并不完美,但它优于人类——这很可能得益于其能够访问海量知识,以及概念加工对人类而言较为费力(Connell and Lynott, 2012)。最后,我们估计了 GPT-3 在多大程度上是在对世界进行推理,而非机械复述其训练数据。我们发现,GPT-3 的输出通常与大型网络语料库存在显著重叠,但机械复述策略对于新颖名词复合词而言收益较小。
英文摘要:
Noun compound interpretation is the task of expressing a noun compound (e.g. chocolate bunny) in a free-text paraphrase that makes the relationship between the constituent nouns explicit (e.g. bunny-shaped chocolate). We propose modifications to the data and evaluation setup of the standard task (Hendrickx et al., 2013), and show that GPT-3 solves it almost perfectly. We then investigate the task of noun compound conceptualization, i.e. paraphrasing a novel or rare noun compound. E.g., chocolate crocodile is a crocodile-shaped chocolate. This task requires creativity, commonsense, and the ability to generalize knowledge about similar concepts. While GPT-3's performance is not perfect, it is better than that of humans -- likely thanks to its access to vast amounts of knowledge, and because conceptual processing is effortful for people (Connell and Lynott, 2012). Finally, we estimate the extent to which GPT-3 is reasoning about the world vs. parroting its training data. We find that the outputs from GPT-3 often have significant overlap with a large web corpus, but that the parroting strategy is less beneficial for novel noun compounds.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。