arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.00560cs.SDeess.AS

编辑内容,保留声学:基于自一致性奖励的不可感知文本语音编辑

Edit Content, Preserve Acoustics: Imperceptible Text-Based Speech Editing via Self-Consistency Rewards

The State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences(多模态人工智能系统国家重点实验室,自动化研究所,中国科学院);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Department of Automation, Tsinghua University(清华大学自动化系);BNRist, Tsinghua University(清华大学BNRist)

Yong Ren, Jiangyan Yi, Jianhua Tao, Tao Wang, Le Xu, Zhengqi Wen

AI总结:

提出一种在稳定语义空间中编辑内容、通过流匹配解码器保持声学连续性的框架,并利用自一致性奖励组相对策略优化实现不可感知的文本语音编辑。

摘要:

不可感知的基于文本的语音编辑通过转录操作修改口语内容,同时保持声学连续性。先前的声学空间方法存在内容-风格纠缠,导致生成不稳定和边界伪影。我们引入了一个以“编辑内容,保留声学”原则为指导的框架。编辑在稳定的语义空间中进行,而声学实现由流匹配解码器处理。为了确保感知一致性,我们提出了自一致性奖励组相对策略优化,该优化利用预训练的文本到语音模型作为隐式评判器,并结合可理解性和持续时间约束。实验表明,在可理解性、鲁棒性和感知质量方面,该方法持续优于最先进的自回归和非自回归基线。

英文摘要:

Imperceptible text-based speech editing modifies spoken content through transcript manipulation while preserving acoustic continuity. Prior acoustic-space approaches suffer from content-style entanglement, causing unstable generation and boundary artifacts. We introduce a framework guided by the principle of "Edit Content, Preserve Acoustics". Editing is conducted in a stable semantic space, while acoustic realization is handled by a Flow Matching decoder. To ensure perceptual consistency, we propose Self-Consistency Rewards Group Relative Policy Optimization, which leverages a pre-trained Text-to-Speech model as an implicit critic, together with intelligibility and duration constraints. Experiments demonstrate consistent improvements over state-of-the-art autoregressive and non-autoregressive baselines in intelligibility, robustness, and perceptual quality.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑