基于否定实例的端到端否定解决方案评估
Negation-Instance Based Evaluation of End-to-End Negation Resolution
摘要:
在本文中,我们重新审视了否定消解的任务,其中包括线索检测(例如,“不”,“从来没有”)和范围解析的子任务。在以前的共享任务的背景下,已经提出了各种评价指标。随后的工作通常使用这些的不同子集,包括变体和自定义实现,使得系统之间的有意义的比较变得困难。从语言学的角度和下游的角度来审视这个问题,我们在这里主张一个基于否定实例的方法来评估否定解决。我们提出的指标对应于对每个实例分数的期望,因此可以直观地解释。为了使研究具有可比性,并促进未来的工作,我们提供了一套目前最先进的系统否定决议的三个英语语料库的结果,并公开提供我们的评估脚本的实施。
英文摘要:
In this paper, we revisit the task of negation resolution, which includes the subtasks of cue detection (e.g. "not", "never") and scope resolution. In the context of previous shared tasks, a variety of evaluation metrics have been proposed. Subsequent works usually use different subsets of these, including variations and custom implementations, rendering meaningful comparisons between systems difficult. Examining the problem both from a linguistic perspective and from a downstream viewpoint, we here argue for a negation-instance based approach to evaluating negation resolution. Our proposed metrics correspond to expectations over per-instance scores and hence are intuitively interpretable. To render research comparable and to foster future work, we provide results for a set of current state-of-the-art systems for negation resolution on three English corpora, and make our implementation of the evaluation scripts publicly available.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。