MAVEN-Fact：大规模事件事实性检测数据集
MAVEN-Fact: A Large-scale Event Factuality Detection Dataset
AI总结：
针对事件事实性检测缺乏大规模数据的问题，本文提出基于MAVEN的MAVEN-Fact数据集，含112,276个事件标注，实验证明其挑战性，并发现事件论元和关系对微调模型有效但对LLM无效，且有助于缓解LLM幻觉。
摘要：
事件事实性检测（EFD）任务旨在判断文本事件的事实性，即对事件是事实、可能性还是不可能性进行分类，这对于忠实地理解和利用事件知识至关重要。然而，由于缺乏高质量的大规模数据，事件事实性检测在事件理解研究中尚未得到充分探索，这限制了EFD社区的发展。为解决这些问题并提供忠实的事件理解，我们基于MAVEN数据集引入了MAVEN-Fact，一个大规模、高质量的EFD数据集。MAVEN-Fact包含112,276个事件的事实性标注，使其成为最大的EFD数据集。大量实验表明，MAVEN-Fact对传统微调模型和大语言模型（LLMs）均具有挑战性。得益于MAVEN中事件论元和关系的全面标注，MAVEN-Fact还支持进一步的分析，我们发现采用事件论元和关系有助于微调模型的事件事实性检测，但对LLMs无益。此外，我们初步研究了事件事实性检测的一个应用案例，发现它有助于缓解LLMs中与事件相关的幻觉。我们的数据集和代码可从https://github.com/lcy2723/MAVEN-FACT获取。
英文摘要：
Event Factuality Detection (EFD) task determines the factuality of textual events, i.e., classifying whether an event is a fact, possibility, or impossibility, which is essential for faithfully understanding and utilizing event knowledge. However, due to the lack of high-quality large-scale data, event factuality detection is under-explored in event understanding research, which limits the development of EFD community. To address these issues and provide faithful event understanding, we introduce MAVEN-Fact, a large-scale and high-quality EFD dataset based on the MAVEN dataset. MAVEN-Fact includes factuality annotations of 112,276 events, making it the largest EFD dataset. Extensive experiments demonstrate that MAVEN-Fact is challenging for both conventional fine-tuned models and large language models (LLMs). Thanks to the comprehensive annotations of event arguments and relations in MAVEN, MAVEN-Fact also supports some further analyses and we find that adopting event arguments and relations helps in event factuality detection for fine-tuned models but does not benefit LLMs. Furthermore, we preliminarily study an application case of event factuality detection and find it helps in mitigating event-related hallucination in LLMs. Our dataset and codes can be obtained from \url{https://github.com/lcy2723/MAVEN-FACT}
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
