一种统一的神经编码语言模型用于选择性可编辑的文本到语音生成
A Unified Neural Codec Language Model for Selective Editable Text to Speech Generation
AI总结：
SpeechEdit是一种通过选择性控制机制实现灵活属性控制的神经编码语言模型，用于可编辑的文本到语音生成。
摘要：
神经编码语言模型通过完全模仿短语音提示的声学特征，实现了令人印象深刻的零样本文本到语音（TTS）生成，包括音色、语调和副语言信息。然而，这种整体模仿限制了其隔离和控制个别属性的能力。在本文中，我们提出了一种统一的编码语言模型SpeechEdit，该模型通过选择性控制机制扩展了零样本TTS。默认情况下，SpeechEdit会重现从语音提示推断出的完整声学特征，但仅根据显式控制指令选择性地覆盖特定属性。为了实现可控建模，SpeechEdit是在我们新构建的LibriEdit数据集上训练的，该数据集提供了从LibriHeavy衍生出的delta（差分）训练对。实验结果表明，我们的方法在保持自然性和鲁棒性的同时，提供了灵活且局部化的所需属性控制。音频样本可在https://speech-editing.github.io/speech-editing/上获取。
英文摘要：
Neural codec language models achieve impressive zero-shot Text-to-Speech (TTS) by fully imitating the acoustic characteristics of a short speech prompt, including timbre, prosody, and paralinguistic information. However, such holistic imitation limits their ability to isolate and control individual attributes. In this paper, we present a unified codec language model SpeechEdit that extends zero-shot TTS with a selective control mechanism. By default, SpeechEdit reproduces the complete acoustic profile inferred from the speech prompt, but it selectively overrides only the attributes specified by explicit control instructions. To enable controllable modeling, SpeechEdit is trained on our newly constructed LibriEdit dataset, which provides delta (difference-aware) training pairs derived from LibriHeavy. Experimental results show that our approach maintains naturalness and robustness while offering flexible and localized control over desired attributes. Audio samples are available at https://speech-editing.github.io/speech-editing/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
