arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2104.07555cs.CL
收藏

Data-QuestEval:一种无参考的数据到文本语义评价度量

Data-QuestEval: A Referenceless Metric for Data-to-Text Semantic Evaluation

Sorbonne Université(索邦大学);CNRS(法国国家科学研究中心);LIP6(巴黎第六大学计算机科学实验室);BNP Paribas(法国巴黎银行);reciTAL;Criteo AI Lab(Criteo人工智能实验室)

Clément Rebuffel, Thomas Scialom, Laure Soulier, Benjamin Piwowarski, Sylvain Lamprier, Jacopo Staiano, Geoffrey Scoutheeten, Patrick Gallinari

摘要:

QuestEval是文本到文本任务中使用的无参考指标,通过自动提问和回答问题,将生成的摘要直接与源文本进行比较。它的适应数据到文本的任务是不简单的,因为它需要多模式的问题生成和搜索系统上考虑的任务,这是很少可用的。为此,我们提出了一种方法来建立合成多模态语料库,使训练多模态组件的数据QuestEval度量。由此产生的度量是无参考和多模态的;它在WebNLG和WikiBio基准测试中获得了与人类判断的最新相关性。作为QuestEval项目的一部分,我们提供data-QuestEval的代码和模型用于可重复性目的。

英文摘要:

QuestEval is a reference-less metric used in text-to-text tasks, that compares the generated summaries directly to the source text, by automatically asking and answering questions. Its adaptation to Data-to-Text tasks is not straightforward, as it requires multimodal Question Generation and Answering systems on the considered tasks, which are seldom available. To this purpose, we propose a method to build synthetic multimodal corpora enabling to train multimodal components for a data-QuestEval metric. The resulting metric is reference-less and multimodal; it obtains state-of-the-art correlations with human judgment on the WebNLG and WikiBio benchmarks. We make data-QuestEval's code and models available for reproducibility purpose, as part of the QuestEval project.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑