arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.15696cs.AIcs.CL
收藏

MixRED:一个混合语言关系抽取数据集

MixRED: A Mix-lingual Relation Extraction Dataset

National Key Laboratory for Novel Software Technology, Nanjing University(南京大学计算机软件新技术全国重点实验室);School of Artificial Intelligence, Nanjing University(南京大学人工智能学院)

Lingxing Kong, Yougang Chu, Zheng Ma, Jianbing Zhang, Liang He, Jiajun Chen

AI总结:

本文针对混合语言场景下的关系抽取研究空白,提出了新任务MixRE并构建了人工标注数据集MixRED,同时评估了监督模型与大语言模型的表现,揭示了其优势、局限及性能提升方向。

摘要:

关系抽取是自然语言处理领域的一项关键任务,具有众多实际应用。现有研究主要关注单语关系抽取或关系抽取的跨语言增强。然而,对于混合语言(或语码转换)场景下的关系抽取,仍存在显著的研究空白,在这种场景中,人们在句子中混合不同语言的内容,生成混合语言内容。由于缺乏专用数据集,现有关系抽取模型在此类场景中的有效性很大程度上未被探索。为解决这一问题,我们引入了一项考虑混合语言场景下关系抽取的新任务,称为MixRE,并构建了人工标注的数据集MixRED来支持该任务。除构建MixRED数据集外,我们还在MixRED上评估了最先进的监督模型和大语言模型(LLMs),揭示了它们在混合语言场景中各自的优势和局限性。此外,我们深入探讨了影响MixRE任务中模型性能的因素,并揭示了增强监督模型和LLMs在此新任务中性能的有前景的方向。

英文摘要:

Relation extraction is a critical task in the field of natural language processing with numerous real-world applications. Existing research primarily focuses on monolingual relation extraction or cross-lingual enhancement for relation extraction. Yet, there remains a significant gap in understanding relation extraction in the mix-lingual (or code-switching) scenario, where individuals intermix contents from different languages within sentences, generating mix-lingual content. Due to the lack of a dedicated dataset, the effectiveness of existing relation extraction models in such a scenario is largely unexplored. To address this issue, we introduce a novel task of considering relation extraction in the mix-lingual scenario called MixRE and constructing the human-annotated dataset MixRED to support this task. In addition to constructing the MixRED dataset, we evaluate both state-of-the-art supervised models and large language models (LLMs) on MixRED, revealing their respective advantages and limitations in the mix-lingual scenario. Furthermore, we delve into factors influencing model performance within the MixRE task and uncover promising directions for enhancing the performance of both supervised models and LLMs in this novel task.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑