增强无文本语音到语音翻译中的表现力迁移
Enhancing expressivity transfer in textless speech-to-speech translation
AI总结：
本研究提出一种在离散语音单元级别利用多语言情感嵌入的方法，以增强无文本语音到语音翻译中的表现力迁移，实验证明其优于现有系统。
摘要：
无文本语音到语音翻译系统由于自监督学习技术的整合而迅速发展。然而，现有的最先进系统在准确捕捉和跨语言迁移表现力方面仍存在不足。表现力在传达情感、细微差别和文化微妙性方面起着至关重要的作用，从而增强跨语言的交流。为解决这一问题，本研究提出了一种新颖的方法，该方法在离散语音单元级别运作，并利用多语言情感嵌入来捕捉与语言无关的信息。具体而言，我们展示了这些嵌入如何被有效用于预测目标语言中语音单元的基频和时长。通过在法语到英语翻译任务上进行的客观和主观实验，我们的研究结果突显了与当前最先进系统相比，我们的方法所实现的表现力迁移的优越性。
英文摘要：
Textless speech-to-speech translation systems are rapidly advancing, thanks to the integration of self-supervised learning techniques. However, existing state-of-the-art systems fall short when it comes to capturing and transferring expressivity accurately across different languages. Expressivity plays a vital role in conveying emotions, nuances, and cultural subtleties, thereby enhancing communication across diverse languages. To address this issue this study presents a novel method that operates at the discrete speech unit level and leverages multilingual emotion embeddings to capture language-agnostic information. Specifically, we demonstrate how these embeddings can be used to effectively predict the pitch and duration of speech units in the target language. Through objective and subjective experiments conducted on a French-to-English translation task, our findings highlight the superior expressivity transfer achieved by our approach compared to current state-of-the-art systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
