Vox-Evaluator：通过多层级评估器增强零样本TTS的稳定性与保真度
Vox-Evaluator: Enhancing Stability and Fidelity for Zero-shot TTS with A Multi-Level Evaluator
AI总结：
本文提出Vox-Evaluator多层级评估器，通过识别并纠正错误语音片段及偏好对齐，增强零样本TTS的稳定性和保真度。
摘要：
由语言模型、扩散模型和掩码生成驱动的零样本文本到语音（TTS）技术近期取得了显著进展，在语音合成中实现了令人印象深刻的自然度。然而，稳定性和保真度仍是关键挑战，表现为发音错误、可闻噪声和质量下降。为解决这些问题，我们引入了Vox-Evaluator，一种多层级评估器，旨在指导TTS系统中错误语音片段的纠正和偏好对齐。它能够识别错误片段的时间边界，并对生成的语音提供整体质量评估。具体而言，为了细化错误片段并增强零样本TTS模型的鲁棒性，我们提出利用评估器自动识别声学错误，掩蔽错误片段，最后基于正确部分重新生成语音条件。此外，从Vox-Evaluator获得的细粒度信息可以指导TTS模型的偏好对齐，从而减少语音合成中的不良案例。由于缺乏适合Vox-Evaluator的训练数据集，我们还构建了一个合成文本-语音数据集，并标注了细粒度的发音错误或音频质量问题。实验结果表明，所提出的Vox-Evaluator通过语音纠正机制和偏好优化，在增强TTS系统的稳定性和保真度方面具有有效性。演示已展示。
英文摘要：
Recent advances in zero-shot text-to-speech (TTS), driven by language models, diffusion models and masked generation, have achieved impressive naturalness in speech synthesis. Nevertheless, stability and fidelity remain key challenges, manifesting as mispronunciations, audible noise, and quality degradation. To address these issues, we introduce Vox-Evaluator, a multi-level evaluator designed to guide the correction of erroneous speech segments and preference alignment for TTS systems. It is capable of identifying the temporal boundaries of erroneous segments and providing a holistic quality assessment of the generated speech. Specifically, to refine erroneous segments and enhance the robustness of the zero-shot TTS model, we propose to automatically identify acoustic errors with the evaluator, mask the erroneous segments, and finally regenerate speech conditioning on the correct portions. In addition, the fine-gained information obtained from Vox-Evaluator can guide the preference alignment for TTS model, thereby reducing the bad cases in speech synthesis. Due to the lack of suitable training datasets for the Vox-Evaluator, we also constructed a synthesized text-speech dataset annotated with fine-grained pronunciation errors or audio quality issues. The experimental results demonstrate the effectiveness of the proposed Vox-Evaluator in enhancing the stability and fidelity of TTS systems through the speech correction mechanism and preference optimization. The demos are shown.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
