多说话人文本到语音中口音与说话人表征的解耦
DART: Disentanglement of Accent and Speaker Representation in Multispeaker Text-to-Speech
AI总结：
本文提出利用多级变分自编码器与向量量化解耦说话人和口音表征，以提升多说话人语音合成的灵活性与个性化控制能力。
摘要：
文本到语音（TTS）系统的最新进展已能根据文本输入生成自然且富有表现力的语音。带口音的TTS旨在通过使合成语音更贴近少数群体听众来提升用户体验，并在各种应用和场景中发挥作用。通过允许用户任意组合说话人身份与口音，语音合成可以进一步变得更加灵活，从而产生广泛的个性化语音输出。当前模型难以解耦说话人和口音表征，导致在保持相同说话人特征的同时难以准确模仿不同口音。我们提出一种新方法，利用多级变分自编码器（ML-VAE）和向量量化（VQ）来解耦说话人和口音表征，以提高语音合成的灵活性和个性化程度。我们提出的方法解决了有效分离说话人与口音特征的挑战，从而能够对合成语音进行更细粒度的控制。代码和语音样本已公开。
英文摘要：
Recent advancements in Text-to-Speech (TTS) systems have enabled the generation of natural and expressive speech from textual input. Accented TTS aims to enhance user experience by making the synthesized speech more relatable to minority group listeners, and useful across various applications and context. Speech synthesis can further be made more flexible by allowing users to choose any combination of speaker identity and accent, resulting in a wide range of personalized speech outputs. Current models struggle to disentangle speaker and accent representation, making it difficult to accurately imitate different accents while maintaining the same speaker characteristics. We propose a novel approach to disentangle speaker and accent representations using multi-level variational autoencoders (ML-VAE) and vector quantization (VQ) to improve flexibility and enhance personalization in speech synthesis. Our proposed method addresses the challenge of effectively separating speaker and accent characteristics, enabling more fine-grained control over the synthesized speech. Code and speech samples are publicly available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
