在生成上下文中寻找意义:人类与语言模型的视角
Finding Sense in Nonsense with Generated Contexts: Perspectives from Humans and Language Models
AI总结:
本文通过人类和语言模型对五个语义偏差数据集中的句子进行评估,探讨了如何区分异常句子和无意义句子,并发现语言模型在生成合理上下文方面表现出色。
摘要:
无意义和异常的句子在计算语义解释模型的发展中起到了关键作用。一个核心挑战是区分仅仅是异常(但可以在上下文中解释)和真正无意义的内容。然而,不清楚(a)现有数据集中的无意义程度,以及(b)LLMs能否做出这种区分。在本文中,我们通过收集人类评估者和LLMs对五种语义偏差数据集中的句子(包括无上下文和有上下文的情况)的可理解性判断来回答这两个问题。我们发现,评估者认为大多数句子仅是异常,只有少数被认为是真正的无意义。我们还显示,LLMs在为异常情况生成合理的上下文方面具有显著的能力。
英文摘要:
Nonsensical and anomalous sentences have been instrumental in the development of computational models of semantic interpretation. A core challenge is to distinguish between what is merely anomalous (but can be interpreted given a supporting context) and what is truly nonsensical. However, it is unclear (a) how nonsensical, rather than merely anomalous, existing datasets are; and (b) how well LLMs can make this distinction. In this paper, we answer both questions by collecting sensicality judgments from human raters and LLMs on sentences from five semantically deviant datasets: both context-free and when providing a context. We find that raters consider most sentences at most anomalous, and only a few as properly nonsensical. We also show that LLMs are substantially skilled in generating plausible contexts for anomalous cases.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。