arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2503.23848cs.CL

SpeechDialogueFactory:生成高质量语音对话数据以加速您的Speech-LLM开发

SpeechDialogueFactory: Generating High-Quality Speech Dialogue Data to Accelerate Your Speech-LLM Development

Monash University(莫纳什大学);MBZUAI(穆罕默德·本·扎耶德人工智能大学)

Minghan Wang, Ye Bai, Yuxia Wang, Thuy-Trang Vu, Ehsan Shareghi, Gholamreza Haffari

AI总结:

针对语音对话数据获取成本高、真实性不足的问题,提出开源框架SpeechDialogueFactory,通过元数据生成、对话脚本编写、副语言语音模拟及语音克隆合成等流水线,高效生成媲美人工录音的高质量语音对话数据,加速Speech-LLM开发。

摘要:

高质量语音对话数据集对Speech-LLM开发至关重要,但现有获取方法存在显著局限。人工录音成本高且涉及隐私问题,而合成方法常缺乏对话真实性。为应对这些挑战,我们提出了SpeechDialogueFactory,这是一个能高效生成自然语音对话的生产就绪框架。该方案采用包含元数据生成、对话脚本编写、副语言丰富的语音模拟及带语音克隆的自然语音合成的完整流水线。此外,系统提供用于详细样本检查的交互式UI和高吞吐量批量合成模式。评估表明,系统生成的对话质量可与人工录音相媲美,同时显著降低生产成本。我们将此项工作作为开源工具包发布,并附带英文和中文示例数据集,赋能Speech-LLM研发。

英文摘要:

High-quality speech dialogue datasets are crucial for Speech-LLM development, yet existing acquisition methods face significant limitations. Human recordings incur high costs and privacy concerns, while synthetic approaches often lack conversational authenticity. To address these challenges, we introduce \textsc{SpeechDialogueFactory}, a production-ready framework for generating natural speech dialogues efficiently. Our solution employs a comprehensive pipeline including metadata generation, dialogue scripting, paralinguistic-enriched utterance simulation, and natural speech synthesis with voice cloning. Additionally, the system provides an interactive UI for detailed sample inspection and a high-throughput batch synthesis mode. Evaluations show that dialogues generated by our system achieve a quality comparable to human recordings while significantly reducing production costs. We release our work as an open-source toolkit, alongside example datasets available in English and Chinese, empowering researchers and developers in Speech-LLM research and development.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑