AMR规范化方法用于更公平的评估
AMR Normalization for Fairer Evaluation
摘要：
意义表征（AMR; Banarescu等人，2013）将句子的含义编码为有向图，而Scatch（Cai和Knight，2013）是评估AMR图的主要指标。然而，Smatch不知道AMR规范允许的图结构中的一些意义等价变化，并对表现出这些变化的AMR给出不同的分数。在本文中，我提出了四个规范化的方法，以帮助确保概念上等同的AMR被评估为等同的。有和没有规范化的等效AMR看起来完全不同-将一个黄金语料库与单独的关系具体化本身进行比较，产生了25个Smatch点的差异，这表明两个系统的输出在没有规范化的情况下可能无法直接比较。本文中描述的算法是在现有的AMR开源Python工具包上实现的，并将在相同的许可证下发布。
英文摘要：
Meaning Representation (AMR; Banarescu et al., 2013) encodes the meaning of sentences as a directed graph and Smatch (Cai and Knight, 2013) is the primary metric for evaluating AMR graphs. Smatch, however, is unaware of some meaning-equivalent variations in graph structure allowed by the AMR Specification and gives different scores for AMRs exhibiting these variations. In this paper I propose four normalization methods for helping to ensure that conceptually equivalent AMRs are evaluated as equivalent. Equivalent AMRs with and without normalization can look quite different---comparing a gold corpus to itself with relation reification alone yields a difference of 25 Smatch points, suggesting that the outputs of two systems may not be directly comparable without normalization. The algorithms described in this paper are implemented on top of an existing open-source Python toolkit for AMR and will be released under the same license.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
