StyleS2ST：直接语音到语音翻译的零样本风格迁移
StyleS2ST: Zero-shot Style Transfer for Direct Speech-to-speech Translation
AI总结：
针对直接语音到语音翻译忽略风格迁移的问题，提出StyleS2ST模型，利用平行语料和TTS数据增强实现跨语言风格迁移，在零样本场景中取得良好效果。
摘要：
直接语音到语音翻译（S2ST）因其相比级联S2ST具有诸多优势而逐渐流行。然而，当前研究主要关注语义翻译的准确性，忽略了从源语言到目标语言的语音风格迁移。缺乏高保真度的表现力平行数据使得这种风格迁移具有挑战性，尤其是在更实际的零样本场景中。为解决此问题，我们首先使用多语言多说话人文本到语音合成（TTS）系统构建平行语料库，然后基于直接S2ST系统框架上的风格适配器，提出具有跨语言语音风格迁移能力的StyleS2ST模型。通过平行语料库训练和非平行TTS数据增强，实现声学模型的连续风格空间建模，StyleS2ST捕获从源语言到目标语言的跨语言声学特征映射。实验表明，StyleS2ST在集内和集外零样本场景中均实现了良好的风格相似性和自然度。
英文摘要：
Direct speech-to-speech translation (S2ST) has gradually become popular as it has many advantages compared with cascade S2ST. However, current research mainly focuses on the accuracy of semantic translation and ignores the speech style transfer from a source language to a target language. The lack of high-fidelity expressive parallel data makes such style transfer challenging, especially in more practical zero-shot scenarios. To solve this problem, we first build a parallel corpus using a multi-lingual multi-speaker text-to-speech synthesis (TTS) system and then propose the StyleS2ST model with cross-lingual speech style transfer ability based on a style adaptor on a direct S2ST system framework. Enabling continuous style space modeling of an acoustic model through parallel corpus training and non-parallel TTS data augmentation, StyleS2ST captures cross-lingual acoustic feature mapping from the source to the target language. Experiments show that StyleS2ST achieves good style similarity and naturalness in both in-set and out-of-set zero-shot scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
