DS-TTS：基于动态双风格特征调制的语音片段零样本说话人风格自适应
DS-TTS: Zero-Shot Speaker Style Adaptation from Voice Clips via Dynamic Dual-Style Feature Modulation
AI总结：
DS-TTS 提出双风格编码网络与动态生成器网络，通过风格门控-电影机制融合互补风格特征并动态适应输入长度，在 VCTK 上显著提升零样本语音克隆的词错误率与说话人相似度。
摘要：
近年来，文本到语音（TTS）技术的进步推动了对个性化音频合成日益增长的需求。零样本语音克隆作为一项专门的 TTS 任务，旨在仅使用单个音频样本和任意文本即可合成目标说话人的声音，而无需在训练阶段事先接触该说话人。该过程采用模式识别技术来分析和复现说话人独特的声学特征。尽管已取得进展，但在适应未见说话人的声音风格方面仍存在挑战，这凸显了 TTS 系统在保持自然度、表现力和说话人保真度的同时，难以泛化处理多样化声音的问题。为解决未见说话人风格自适应的挑战，我们提出了 DS-TTS，这是一种旨在增强对多样化、前所未闻声音合成能力的新方法。我们方法的核心是双风格编码网络（DuSEN），其中两个不同的风格编码器捕获说话人声音身份的互补方面。这些特定于说话人的风格向量通过风格门控-电影（SGF）机制无缝集成到动态生成器网络（DyGN）中，从而能够更准确、更具表现力地复现未见说话人独特的声音特征。此外，我们引入动态生成器网络来解决因句子长度变化而产生的合成问题。通过动态适应输入长度，该组件确保了在不同文本输入和说话人风格下的稳健性能，显著提升了模型以更自然、更具表现力的方式泛化到未见说话人的能力。在 VCTK 数据集上的实验评估表明，与现有最先进模型相比，DS-TTS 在语音克隆任务中展现出优越的整体性能，在词错误率和说话人相似度方面均取得了显著改进。
英文摘要：
Recent advancements in text-to-speech (TTS) technology have increased demand for personalized audio synthesis. Zero-shot voice cloning, a specialized TTS task, aims to synthesize a target speaker's voice using only a single audio sample and arbitrary text, without prior exposure to the speaker during training. This process employs pattern recognition techniques to analyze and replicate the speaker's unique vocal features. Despite progress, challenges remain in adapting to the vocal style of unseen speakers, highlighting difficulties in generalizing TTS systems to handle diverse voices while maintaining naturalness, expressiveness, and speaker fidelity. To address the challenges of unseen speaker style adaptation, we propose DS-TTS, a novel approach aimed at enhancing the synthesis of diverse, previously unheard voices. Central to our method is a Dual-Style Encoding Network (DuSEN), where two distinct style encoders capture complementary aspects of a speaker's vocal identity. These speaker-specific style vectors are seamlessly integrated into the Dynamic Generator Network (DyGN) via a Style Gating-Film (SGF) mechanism, enabling more accurate and expressive reproduction of unseen speakers' unique vocal characteristics. In addition, we introduce a Dynamic Generator Network to tackle synthesis issues that arise with varying sentence lengths. By dynamically adapting to the length of the input, this component ensures robust performance across diverse text inputs and speaker styles, significantly improving the model's ability to generalize to unseen speakers in a more natural and expressive manner. Experimental evaluations on the VCTK dataset suggest that DS-TTS demonstrates superior overall performance in voice cloning tasks compared to existing state-of-the-art models, showing notable improvements in both word error rate and speaker similarity.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
