MSM-VC：基于多尺度风格建模的非平行语音转换高保真源风格迁移
MSM-VC: High-fidelity Source Style Transfer for Non-Parallel Voice Conversion by Multi-scale Style Modeling
AI总结：
提出MSM-VC多尺度风格建模方法，通过帧级、局部和全局风格表示及显式约束模块，实现非平行语音转换中源风格的高保真迁移，同时保持语音质量和说话人相似度。
摘要：
在语音转换（VC）任务中，除了将源语音的语言内容传递到转换语音之外，保持源语音的说话风格也扮演着重要角色，这在许多源语音高度富有表现力的场景中至关重要，例如配音和数据增强。以往的工作通常采用从源语音中提取的显式韵律特征或固定长度的风格嵌入来建模源语音的说话风格，这不足以实现全面的风格建模和目标说话人音色保持。受人类语音风格多尺度特性的启发，本文提出了一种用于语音转换任务的多尺度风格建模方法，称为MSM-VC。MSM-VC从不同层级对源语音的说话风格进行建模。为了有效传递说话风格同时防止音色从源语音泄漏到转换语音，每个层级的风格都由特定的表示来建模。具体而言，韵律特征、预训练ASR模型的瓶颈特征以及通过自监督策略训练的模型提取的特征分别用于建模帧级、局部和全局风格。此外，为了平衡源风格建模和目标说话人音色保持的性能，MSM-VC引入了一个由预训练语音情感识别模型和说话人分类器组成的显式约束模块。该显式约束模块还使得在训练过程中模拟风格迁移推理过程成为可能，以提高解耦能力并缓解训练与推理之间的不匹配。在高度富有表现力的语音语料库上进行的实验表明，MSM-VC在建模源语音风格方面优于最先进的语音转换方法，同时保持了良好的语音质量和说话人相似度。
英文摘要：
In addition to conveying the linguistic content from source speech to converted speech, maintaining the speaking style of source speech also plays an important role in the voice conversion (VC) task, which is essential in many scenarios with highly expressive source speech, such as dubbing and data augmentation. Previous work generally took explicit prosodic features or fixed-length style embedding extracted from source speech to model the speaking style of source speech, which is insufficient to achieve comprehensive style modeling and target speaker timbre preservation. Inspired by the style's multi-scale nature of human speech, a multi-scale style modeling method for the VC task, referred to as MSM-VC, is proposed in this paper. MSM-VC models the speaking style of source speech from different levels. To effectively convey the speaking style and meanwhile prevent timbre leakage from source speech to converted speech, each level's style is modeled by specific representation. Specifically, prosodic features, pre-trained ASR model's bottleneck features, and features extracted by a model trained with a self-supervised strategy are adopted to model the frame, local, and global-level styles, respectively. Besides, to balance the performance of source style modeling and target speaker timbre preservation, an explicit constraint module consisting of a pre-trained speech emotion recognition model and a speaker classifier is introduced to MSM-VC. This explicit constraint module also makes it possible to simulate the style transfer inference process during the training to improve the disentanglement ability and alleviate the mismatch between training and inference. Experiments performed on the highly expressive speech corpus demonstrate that MSM-VC is superior to the state-of-the-art VC methods for modeling source speech style while maintaining good speech quality and speaker similarity.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
