利用文本转语音与大语言模型生成对话式语音识别数据
Generating Data with Text-to-Speech and Large-Language Models for Conversational Speech Recognition
AI总结：
针对多说话人对话语音识别领域数据获取难的问题，提出结合LLM内容生成与多说话人对话式TTS的合成数据流水线，经Whisper模型微调验证，性能优于经典多说话人生成方法。
摘要：
当前，许多语音处理任务的常用方法是依托大规模预训练模型，针对特定应用在领域内数据上进行微调。但受隐私问题与标注成本限制，即便获取少量此类数据也困难重重，在敏感领域和对话式语音场景中尤为突出。为解决这一问题，已有研究采用单说话人数据集生成合成数据，但针对多说话人场景，这类方法往往需要大量人工操作，且易出现领域不匹配问题。本研究提出一套面向多说话人对话式自动语音识别（ASR）的合成数据生成流水线，借助大语言模型（LLM）生成内容，利用多说话人对话式文本转语音（TTS）模型合成语音。我们分别使用领域内数据与生成的合成数据，对Whisper ASR模型进行微调，在电话对话与远场对话语音场景下开展评估。结果表明，所提方法的性能显著优于使用外部非对话语音数据集的经典多说话人生成方法。
英文摘要：
Currently, a common approach in many speech processing tasks is to leverage large scale pre-trained models by fine-tuning them on in-domain data for a particular application. Yet obtaining even a small amount of such data can be problematic, especially for sensitive domains and conversational speech scenarios, due to both privacy issues and annotation costs. To address this, synthetic data generation using single speaker datasets has been employed. Yet, for multi-speaker cases, such an approach often requires extensive manual effort and is prone to domain mismatches. In this work, we propose a synthetic data generation pipeline for multi-speaker conversational ASR, leveraging a large language model (LLM) for content creation and a conversational multi-speaker text-to-speech (TTS) model for speech synthesis. We conduct evaluation by fine-tuning the Whisper ASR model for telephone and distant conversational speech settings, using both in-domain data and generated synthetic data. Our results show that the proposed method is able to significantly outperform classical multi-speaker generation approaches that use external, non-conversational speech datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
