arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.02586cs.CL
收藏

提升零样本事件检测中的事件定义遵循能力

Improving Event Definition Following For Zero-Shot Event Detection

Peking University(北京大学);University of California, Los Angeles(加利福尼亚大学洛杉矶分校)

Zefan Cai, Po-Nien Kung, Ashima Suvarna, Mingyu Derek Ma, Hritik Bansal, Baobao Chang, P. Jeffrey Brantingham, Wei Wang, Nanyun Peng

AI总结:

本研究针对零样本事件检测中模型事件定义遵循能力不足的问题,构建了DivED数据集,结合事件本体信息与难负样本微调LLaMA-2-7B,在三个公开基准上性能超越GPT-3.5等SOTA模型。

摘要:

现有的零样本事件检测方法通常在标注了已知事件类型的数据集上训练模型,并用未见过的事件定义进行提示。这些方法虽能取得零星的成功,但总体效果未达预期。在本研究中,我们旨在通过训练模型更好地遵循事件定义来提升零样本事件检测性能。我们假设,多样化的事件类型与定义是模型学习遵循事件定义的关键,而现有的事件抽取数据集专注于为少量事件类型标注大量高质量样本。为验证该假设,我们构建了一个自动生成的Diverse Event Definition(DivED,多样化事件定义)数据集,并开展了对比研究。实验表明,200种的大量事件类型以及多样化的事件定义能显著提升事件抽取性能;另一方面,当每个事件类型的样本超过10个时,性能不会随样本量增加而提升。除了规模扩展,我们在训练中引入了事件本体信息和难负样本,进一步提升了性能。基于这些发现,我们在DivED数据集上微调了LLaMA-2-7B模型,在三个零样本事件检测公开基准上的性能超越了GPT-3.5等SOTA大语言模型。

英文摘要:

Existing approaches on zero-shot event detection usually train models on datasets annotated with known event types, and prompt them with unseen event definitions. These approaches yield sporadic successes, yet generally fall short of expectations. In this work, we aim to improve zero-shot event detection by training models to better follow event definitions. We hypothesize that a diverse set of event types and definitions are the key for models to learn to follow event definitions while existing event extraction datasets focus on annotating many high-quality examples for a few event types. To verify our hypothesis, we construct an automatically generated Diverse Event Definition (DivED) dataset and conduct comparative studies. Our experiments reveal that a large number of event types (200) and diverse event definitions can significantly boost event extraction performance; on the other hand, the performance does not scale with over ten examples per event type. Beyond scaling, we incorporate event ontology information and hard-negative samples during training, further boosting the performance. Based on these findings, we fine-tuned a LLaMA-2-7B model on our DivED dataset, yielding performance that surpasses SOTA large language models like GPT-3.5 across three open benchmarks on zero-shot event detection.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑