所有步骤都同等重要吗？事件必要性检测的基准评测
Are All Steps Equally Important? Benchmarking Essentiality Detection of Events
AI总结：
本文构建了基于WikiHow的（目标，步骤）对语料库并人工标注步骤必要性，通过基准评测发现现有统计模型和预训练语言模型在事件必要性检测上显著落后于人类，揭示了该任务的挑战性。
摘要：
自然语言以不同的粒度表达事件，其中粗粒度事件（目标）可以分解为更细粒度的事件序列（步骤）。理解事件过程中一个关键但被忽视的方面是：并非所有步骤事件对完成目标具有同等重要性。本文通过考察当前模型在多大程度上理解步骤事件相对于目标事件的必要性来填补这一空白。认知研究表明，这种能力使机器能够模拟人类关于日常任务前提条件和必要努力的常识推理。我们贡献了一个高质量的（目标，步骤）对语料库，该语料库收集自社区指南网站WikiHow，并由专家针对目标对步骤的必要性进行了人工标注。较高的标注者间一致性表明人类对事件必要性具有一致的理解。然而，在评估了多个统计模型和大规模预训练语言模型后，我们发现现有方法的表现明显不如人类。这一观察结果凸显了进一步探索这一关键且具有挑战性任务的必要性。数据集和代码可在http://cogcomp.org/page/publication_view/1023获取。
英文摘要：
Natural language expresses events with varying granularities, where coarse-grained events (goals) can be broken down into finer-grained event sequences (steps). A critical yet overlooked aspect of understanding event processes is recognizing that not all step events hold equal importance toward the completion of a goal. In this paper, we address this gap by examining the extent to which current models comprehend the essentiality of step events in relation to a goal event. Cognitive studies suggest that such capability enables machines to emulate human commonsense reasoning about preconditions and necessary efforts of everyday tasks. We contribute a high-quality corpus of (goal, step) pairs gathered from the community guideline website WikiHow, with steps manually annotated for their essentiality concerning the goal by experts. The high inter-annotator agreement demonstrates that humans possess a consistent understanding of event essentiality. However, after evaluating multiple statistical and largescale pre-trained language models, we find that existing approaches considerably underperform compared to humans. This observation highlights the need for further exploration into this critical and challenging task. The dataset and code are available at http://cogcomp.org/page/publication_view/1023.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
