迈向日语全双工口语对话系统
Towards a Japanese Full-duplex Spoken Dialogue System
AI总结：
本文提出首个公开的日语全双工口语对话模型，基于Moshi构建，通过大规模日语口语数据预训练、高质量立体声对话微调及合成数据增强，在自然度和有意义性上超越日语基线。
摘要：
全双工口语对话系统能够建模人类对话中同时发生的双向特征，如语音重叠和反馈语，近年来引起了广泛关注。然而，针对日语的全双工口语对话系统研究仍然有限，关于其日语开发的研究也十分稀缺。在本文中，我们提出了首个公开可用的日语全双工口语对话模型，该模型建立在英语全双工对话模型Moshi之上。我们的模型通过两阶段过程进行训练：首先在大规模日语口语对话数据上进行预训练，随后在高质量立体声口语对话数据上进行微调。我们进一步通过引入由多流文本到语音系统生成的合成对话数据来提升模型性能。评估实验表明，训练后的模型在自然度和有意义性方面均优于日语基线模型。
英文摘要：
Full-duplex spoken dialogue systems, which can model simultaneous bidirectional features of human conversations such as speech overlaps and backchannels, have attracted significant attention recently. However, the study of full-duplex spoken dialogue systems for the Japanese language has been limited, and the research on their development in Japanese remains scarce. In this paper, we present the first publicly available full-duplex spoken dialogue model in Japanese, which is built upon Moshi, a full-duplex dialogue model in English. Our model is trained through a two-stage process: pre-training on a large-scale spoken dialogue data in Japanese, followed by fine-tuning on high-quality stereo spoken dialogue data. We further enhance the model's performance by incorporating synthetic dialogue data generated by a multi-stream text-to-speech system. Evaluation experiments demonstrate that the trained model outperforms Japanese baseline models in both naturalness and meaningfulness.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
