KPQA：一种基于关键词权重的生成式问答度量
KPQA: A Metric for Generative Question Answering Using Keyphrase Weights
摘要：
在生成式问答（GenQA）系统的自动评测中，由于答案的形式不统一，很难对生成答案的正确性进行评估。特别是，广泛使用的n元相似性度量通常无法区分不正确的答案，因为它们同等地考虑所有标记。为了缓解这个问题，我们提出了KPQA度量，一个新的度量GenQA的正确性进行评估。具体来说，我们的新指标通过关键短语预测为每个令牌分配不同的权重，从而判断生成的答案句子是否捕捉到参考答案的关键含义。为了评估我们的指标，我们在两个GenQA数据集上创建了高质量的人类正确性判断。使用我们的人类评估数据集，我们表明，我们提出的指标有一个显着更高的相关性与人类的判断比现有的指标。该代码可在https://github.com/hwanheelee1993/KPQA上获得。
英文摘要：
In the automatic evaluation of generative question answering (GenQA) systems, it is difficult to assess the correctness of generated answers due to the free-form of the answer. Especially, widely used n-gram similarity metrics often fail to discriminate the incorrect answers since they equally consider all of the tokens. To alleviate this problem, we propose KPQA-metric, a new metric for evaluating the correctness of GenQA. Specifically, our new metric assigns different weights to each token via keyphrase prediction, thereby judging whether a generated answer sentence captures the key meaning of the reference answer. To evaluate our metric, we create high-quality human judgments of correctness on two GenQA datasets. Using our human-evaluation datasets, we show that our proposed metric has a significantly higher correlation with human judgments than existing metrics. The code is available at https://github.com/hwanheelee1993/KPQA.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
