RQUGE：通过回答问题来评估问题生成的无参考指标
RQUGE: Reference-Free Metric for Evaluating Question Generation by Answering the Question
AI总结：
本文提出无参考评估指标RQUGE，通过问答模块和跨度评分模块衡量候选问题在上下文中的可回答性，无需人工参考问题，与人类判断相关性更高且更鲁棒，并能通过重排序合成数据提升QA模型域外性能。
摘要：
现有的用于评估自动生成问题质量的指标，如BLEU、ROUGE、BERTScore和BLEURT，通过比较参考问题和预测问题来给出评分，当候选问题与参考问题之间存在相当大的词汇重叠或语义相似性时，就会给出高分。这种方法有两个主要缺点。首先，我们需要昂贵的人工提供的参考问题。其次，它惩罚了那些可能与参考问题没有很高词汇或语义相似性的有效问题。在本文中，我们提出了一种新的指标RQUGE，该指标基于候选问题在给定上下文中的可回答性。该指标由一个问答模块和一个跨度评分模块组成，使用现有文献中的预训练模型，因此无需任何进一步训练即可使用。我们证明了RQUGE在不依赖参考问题的情况下与人类判断具有更高的相关性。此外，RQUGE被证明对几种对抗性破坏更加鲁棒。进一步地，我们说明了通过在由问题生成模型生成并经RQUGE重新排序的合成数据上进行微调，可以显著提高QA模型在域外数据集上的性能。
英文摘要：
Existing metrics for evaluating the quality of automatically generated questions such as BLEU, ROUGE, BERTScore, and BLEURT compare the reference and predicted questions, providing a high score when there is a considerable lexical overlap or semantic similarity between the candidate and the reference questions. This approach has two major shortcomings. First, we need expensive human-provided reference questions. Second, it penalises valid questions that may not have high lexical or semantic similarity to the reference questions. In this paper, we propose a new metric, RQUGE, based on the answerability of the candidate question given the context. The metric consists of a question-answering and a span scorer modules, using pre-trained models from existing literature, thus it can be used without any further training. We demonstrate that RQUGE has a higher correlation with human judgment without relying on the reference question. Additionally, RQUGE is shown to be more robust to several adversarial corruptions. Furthermore, we illustrate that we can significantly improve the performance of QA models on out-of-domain datasets by fine-tuning on synthetic data generated by a question generation model and re-ranked by RQUGE.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
