利用多个TTS目标增强语音到语音翻译
Enhancing Speech-to-Speech Translation with Multiple TTS Targets
AI总结：
本文针对直接语音到语音翻译的数据稀缺问题，提出一种利用不同TTS系统的多个目标语音联合优化S2ST的多任务框架，在Fisher数据集上取得2.8 BLEU提升。
摘要：
众所周知，由于源语音和目标语音的现有平行材料有限，直接语音到语音翻译（S2ST）模型通常面临数据稀缺问题。因此，为了训练直接S2ST系统，以往的工作通常利用文本到语音（TTS）系统，通过扩充来自语音到文本翻译（S2TT）的数据来生成目标语言的样本。然而，关于合成的目标语音如何影响S2ST模型的研究仍然有限。在这项工作中，我们分析了改变合成目标语音对直接S2ST模型的影响。我们发现，简单地组合来自不同TTS系统的目标语音有可能提升S2ST的性能。随后，我们还提出了一种多任务框架，利用来自不同TTS系统的多个目标联合优化S2ST系统。大量实验表明，我们提出的框架在Fisher西班牙语-英语数据集上相比基线模型取得了一致的改进（2.8 BLEU）。
英文摘要：
It has been known that direct speech-to-speech translation (S2ST) models usually suffer from the data scarcity issue because of the limited existing parallel materials for both source and target speech. Therefore to train a direct S2ST system, previous works usually utilize text-to-speech (TTS) systems to generate samples in the target language by augmenting the data from speech-to-text translation (S2TT). However, there is a limited investigation into how the synthesized target speech would affect the S2ST models. In this work, we analyze the effect of changing synthesized target speech for direct S2ST models. We find that simply combining the target speech from different TTS systems can potentially improve the S2ST performances. Following that, we also propose a multi-task framework that jointly optimizes the S2ST system with multiple targets from different TTS systems. Extensive experiments demonstrate that our proposed framework achieves consistent improvements (2.8 BLEU) over the baselines on the Fisher Spanish-English dataset.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
