PaCo：归因于常识知识的前提条件
PaCo: Preconditions Attributed to Commonsense Knowledge
AI总结：
本文提出 PaCo 数据集与三项评估任务，考察语言模型对常识陈述情境性前提条件的推理能力，发现其与人类表现仍有10%–30%差距。
摘要：
人类能够毫不费力地依据常识知识的情境性前提条件进行推理。我们知道玻璃杯可用于喝水，除非玻璃杯破损或水有毒。尽管最先进（SOTA）语言模型（LM）在推断常识知识方面表现出色，但尚不清楚它们是否理解这些情境性前提条件。为弥补这一差距，我们提出了一项利用情境性前提条件进行推理的新型挑战。我们收集了一个名为 PaCo 的数据集，其中包含以自然语言表达的 1.24 万条常识陈述前提条件。基于该数据集，我们设计了三个规范评估任务，并用它们考察现有 LM 理解情境前提条件的能力。结果显示，在这些任务上，机器与人类表现之间存在 10%–30% 的差距，表明基于前提条件的推理仍是一项开放性挑战。
英文摘要：
Humans can seamlessly reason with circumstantial preconditions of commonsense knowledge. We understand that a glass is used for drinking water, unless the glass is broken or the water is toxic. Despite state-of-the-art (SOTA) language models' (LMs) impressive performance on inferring commonsense knowledge, it is unclear whether they understand the circumstantial preconditions. To address this gap, we propose a novel challenge of reasoning with circumstantial preconditions. We collect a dataset, called PaCo, consisting of 12.4 thousand preconditions of commonsense statements expressed in natural language. Based on this dataset, we create three canonical evaluation tasks and use them to examine the capability of existing LMs to understand situational preconditions. Our results reveal a 10-30% gap between machine and human performance on our tasks, which shows that reasoning with preconditions is an open challenge.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
