无需并行数据的多语言表达性语音表征学习用于韵律预测
Learning Multilingual Expressive Speech Representation for Prosody Prediction without Parallel Data
AI总结：
提出基于离散语音单元的多语言情感嵌入方法，通过预测音高和时长实现跨语言情感保持的语音重合成，无需并行数据且优于基线。
摘要：
我们提出一种在离散语音单元层面进行语音到语音情感保持翻译的方法。该方法依赖于以语言无关方式捕捉情感信息的多语言情感嵌入。我们表明，该嵌入可用于预测目标语言中语音单元的音高和时长，从而允许我们用相同的情感内容重新合成源语音信号。我们评估了该方法在英语和法语语音信号上的表现，结果表明它优于不使用情感信息的基线方法，包括当情感嵌入从不同语言中提取时。尽管这项初步研究未直接解决机器翻译问题，但我们的结果证明了该方法在语音重合成背景下进行跨语言情感保持的有效性。
英文摘要：
We propose a method for speech-to-speech emotionpreserving translation that operates at the level of discrete speech units. Our approach relies on the use of multilingual emotion embedding that can capture affective information in a language-independent manner. We show that this embedding can be used to predict the pitch and duration of speech units in a target language, allowing us to resynthesize the source speech signal with the same emotional content. We evaluate our approach to English and French speech signals and show that it outperforms a baseline method that does not use emotional information, including when the emotion embedding is extracted from a different language. Even if this preliminary study does not address directly the machine translation issue, our results demonstrate the effectiveness of our approach for cross-lingual emotion preservation in the context of speech resynthesis.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
