arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2409.00331cs.CLcs.AI
收藏

WikiCausal:面向因果知识图谱构建的语料库与评估框架

WikiCausal: Corpus and Evaluation Framework for Causal Knowledge Graph Construction

IBM Research(IBM研究院)

Oktie Hassanzadeh

AI总结:

本文提出WikiCausal语料库与评估框架,用于从维基百科文章中抽取事件因果关系,并结合Wikidata已有关系和大型语言模型进行自动评估,以有效选择各任务的最佳模型。

摘要:

近年来,人们对通用领域和特定领域因果知识图谱的构建越来越感兴趣。这类知识图谱能够支持因果分析和事件预测的推理,因此在不同领域中具有广泛的应用。尽管在因果知识图谱的自动化构建方面已经取得了很大进展,但对此类解决方案的评估要么集中在低层任务上(例如因果短语抽取),要么依赖于临时评估数据和规模较小的人工评估。本文提出了一个用于因果知识图谱构建的语料库、任务和评估框架。我们的语料库由Wikidata中一组与事件相关概念所对应的维基百科文章组成。任务是从语料库中抽取事件概念之间的因果关系。评估部分利用Wikidata中已有的因果关系来衡量召回率,部分使用Large Language Models来避免人工或众包评估的需要。我们评估了一个用于因果知识图谱构建的流水线,该流水线依赖神经模型进行问答和概念链接,并展示了该语料库和评估框架如何使我们能够有效地为每个任务找到合适的模型。该语料库和评估框架已公开发布。

英文摘要:

Recently, there has been an increasing interest in the construction of general-domain and domain-specific causal knowledge graphs. Such knowledge graphs enable reasoning for causal analysis and event prediction, and so have a range of applications across different domains. While great progress has been made toward automated construction of causal knowledge graphs, the evaluation of such solutions has either focused on low-level tasks (e.g., cause-effect phrase extraction) or on ad hoc evaluation data and small manual evaluations. In this paper, we present a corpus, task, and evaluation framework for causal knowledge graph construction. Our corpus consists of Wikipedia articles for a collection of event-related concepts in Wikidata. The task is to extract causal relations between event concepts from the corpus. The evaluation is performed in part using existing causal relations in Wikidata to measure recall, and in part using Large Language Models to avoid the need for manual or crowd-sourced evaluation. We evaluate a pipeline for causal knowledge graph construction that relies on neural models for question answering and concept linking, and show how the corpus and the evaluation framework allow us to effectively find the right model for each task. The corpus and the evaluation framework are publicly available.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑