基于潜状态空间建模的文本驱动语音转换
Text-Driven Voice Conversion via Latent State-Space Modeling
AI总结：
针对现有文本驱动语音转换方法风格控制灵活性不足的问题，提出基于潜状态空间的LSS-VC方法，通过语音潜流形与自适应跨模态融合实现细粒度风格控制，性能优于现有基线。
摘要：
文本驱动语音转换可借助文本描述定制说话人特征与韵律元素。但现有多数方法高度依赖直接的文本转语音训练，限制了其在精细风格元素或音色特征控制方面的灵活性。本文提出一种用于文本驱动语音转换的新型潜状态空间方法（LSS-VC）。该方法将每条话语视为连续潜空间中不断演化的动力系统。受mamba（一种引入状态空间模型实现高效文本驱动图像风格迁移的模型）启发，我们将一种关联度较低的方法适配应用于语音风格转换。具体而言，我们学习了一个语音潜流形，其中风格与内容可通过文本风格提示独立操控。我们还提出自适应跨模态融合机制，将风格信息注入语音潜表示，实现对说话人身份、语速和重音的可解释细粒度控制。大量实验表明，该方法在主观与客观质量指标上均显著优于近期基线模型，同时具备更平滑的风格过渡、更少的伪影以及更精准的文本风格控制能力。
英文摘要：
Text-driven voice conversion allows customization of speaker characteristics and prosodic elements using textual descriptions. However, most existing methods rely heavily on direct text-to-speech training, limiting their flexibility in controlling nuanced style elements or timbral features. In this paper, we propose a novel \textbf{Latent State-Space} approach for text-driven voice conversion (\textbf{LSS-VC}). Our method treats each utterance as an evolving dynamical system in a continuous latent space. Drawing inspiration from mamba, which introduced a state-space model for efficient text-driven \emph{image} style transfer, we adapt a loosely related methodology for \emph{voice} style transformation. Specifically, we learn a voice latent manifold where style and content can be manipulated independently by textual style prompts. We propose an adaptive cross-modal fusion mechanism to inject style information into the voice latent representation, enabling interpretable and fine-grained control over speaker identity, speaking rate, and emphasis. Extensive experiments show that our approach significantly outperforms recent baselines in both subjective and objective quality metrics, while offering smoother transitions between styles, reduced artifacts, and more precise text-based style control.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
