arXiv 2601.009352026-01-02eess.AScs.AI

通过TTS数据增强改进代码切换语音识别

Improving Code-Switching Speech Recognition with TTS Data Augmentation

Institute for Infocomm Research (I 2 R), A*STAR, Singapore（信息通信研究所（I 2 R），A*STAR，新加坡）；College of Computing and Data Science, Nanyang Technological University, Singapore（计算与数据科学学院，南洋理工大学，新加坡）

Yue Heng Yeo, Yuchen Hu, Shreyas Gopal, Yizhou Peng, Hexin Liu, Eng Siong Chng

备注：: This paper was accepted by APSIPA 2025

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

利用多语言TTS模型生成合成语音数据，有效提升代码切换语音识别的准确率和鲁棒性。

摘要：

对话中代码切换语音的自动语音识别（ASR）仍然具有挑战性，因为现实高质量标注语音数据稀缺。本文探索多语言文本到语音（TTS）模型作为有效的数据增强技术来解决这一短缺。具体来说，我们对SEAME数据集进行微调，以多语言CosyVoice2 TTS模型生成合成的中文-英文代码切换语音，显著增加了可用训练数据的数量和说话者多样性。我们的实验表明，用合成语音增强真实语音可将DevMan上的混合错误率（MER）从12.1%降低到10.1%，将DevSGE上的MER从17.8%降低到16.0%，表明一致的性能提升。这些结果证实了多语言TTS在低资源对话代码切换场景中增强ASR鲁棒性的有效性和实用性。

英文摘要：

Automatic speech recognition (ASR) for conversational code-switching speech remains challenging due to the scarcity of realistic, high-quality labeled speech data. This paper explores multilingual text-to-speech (TTS) models as an effective data augmentation technique to address this shortage. Specifically, we fine-tune the multilingual CosyVoice2 TTS model on the SEAME dataset to generate synthetic conversational Chinese-English code-switching speech, significantly increasing the quantity and speaker diversity of available training data. Our experiments demonstrate that augmenting real speech with synthetic speech reduces the mixed error rate (MER) from 12.1 percent to 10.1 percent on DevMan and from 17.8 percent to 16.0 percent on DevSGE, indicating consistent performance gains. These results confirm that multilingual TTS is an effective and practical tool for enhancing ASR robustness in low-resource conversational code-switching scenarios.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。