Source2Synth：基于真实数据源的合成数据生成与筛选
Source2Synth: Synthetic Data Generation and Curation Grounded in Real Data Sources
AI总结：
针对现有合成数据生成方法质量低或不自然的问题，提出基于真实数据源的可扩展方法Source2Synth，通过可回答性筛选低质量内容并生成带推理步骤的样本，在TQA和MHQA任务上分别提升25.51%和22.57%性能。
摘要：
合成数据生成近年来成为增强大型语言模型（LLMs）能力的一种有前景的方法，无需昂贵的人工标注。然而，现有方法生成的数据往往质量低下或不自然。本文提出Source2Synth，一种基于真实世界数据源的可扩展合成数据生成与筛选方法。Source2Synth以自定义数据源为输入，生成带有中间推理步骤的合成数据样本。我们的方法通过基于可回答性丢弃低质量生成内容来提升数据集质量。我们将该方法应用于两个利用不同类型数据的任务，以证明其通用性：多跳问答（MHQA）——利用文档测试复杂推理能力，以及表格问答（TQA）——利用表格测试工具使用能力。与微调基线相比，我们的方法在WikiSQL上的TQA性能提升25.51%，在HotpotQA上的MHQA性能提升22.57%。
英文摘要：
Synthetic data generation has recently emerged as a promising approach for enhancing the capabilities of large language models (LLMs) without the need for expensive human annotations. However, existing methods often generate data that can be low quality or contrived. In this paper, we introduce Source2Synth, a scalable approach for synthetic data generation and curation that is grounded in real-world data sources. Source2Synth takes as input a custom data source and produces synthetic data examples with intermediate reasoning steps. Our method improves the dataset quality by discarding low-quality generations based on their answerability. We demonstrate the generality of this approach by applying it to two tasks that leverage two different types of data: multi-hop question answering (MHQA), where we test complex reasoning abilities leveraging documents, and tabular question answering (TQA), where we test tool usage leveraging tables. Our method improves performance by 25.51% for TQA on WikiSQL and 22.57% for MHQA on HotpotQA compared to the fine-tuned baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
