Genie：在基于内容的数据集生成中达到人类同等水平
Genie: Achieving Human Parity in Content-Grounded Datasets Generation
AI总结：
本文提出三阶段数据生成方法 Genie，通过内容准备、任务示例生成与质量过滤，为 LFQA、摘要和信息抽取构建高质量合成数据，并证明其训练效果达到或超过人工数据。
摘要：
缺乏用于基于内容的生成任务的高质量数据，已被确定为推进这些任务的主要障碍。为弥补这一差距，我们提出 Genie，这是一种自动生成高质量基于内容的数据的新方法。它包括三个阶段：（a）内容准备；（b）生成：根据内容创建特定任务的示例（例如问答对或摘要）；（c）过滤机制，旨在确保生成数据的质量和忠实性。我们通过为长文本问答（LFQA）、摘要和信息抽取生成三个大规模合成数据 making wishes 来展示该方法。在人工评估中，我们生成的数据被认为自然且质量高。此外，我们将使用我们的数据训练的模型与使用人工编写数据训练的模型进行比较——LFQA 使用 ELI5 和 ASQA，摘要使用 CNN-DailyMail。结果表明，我们的模型与使用人工生成数据训练的模型相当或更优，并且在忠实性方面始终优于它们。最后，我们将该方法应用于创建医疗领域的 LFQA 数据，并将使用该数据训练的模型与使用其他领域数据训练的模型进行了比较。
英文摘要：
The lack of high-quality data for content-grounded generation tasks has been identified as a major obstacle to advancing these tasks. To address this gap, we propose Genie, a novel method for automatically generating high-quality content-grounded data. It consists of three stages: (a) Content Preparation, (b) Generation: creating task-specific examples from the content (e.g., question-answer pairs or summaries). (c) Filtering mechanism aiming to ensure the quality and faithfulness of the generated data. We showcase this methodology by generating three large-scale synthetic data, making wishes, for Long-Form Question-Answering (LFQA), summarization, and information extraction. In a human evaluation, our generated data was found to be natural and of high quality. Furthermore, we compare models trained on our data with models trained on human-written data -- ELI5 and ASQA for LFQA and CNN-DailyMail for Summarization. We show that our models are on par with or outperforming models trained on human-generated data and consistently outperforming them in faithfulness. Finally, we applied our method to create LFQA data within the medical domain and compared a model trained on it with models trained on other domains.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
