面向情感一致的基于文本的语音编辑：引入EmoCorrector与ECD-TSE数据集
Towards Emotionally Consistent Text-Based Speech Editing: Introducing EmoCorrector and The ECD-TSE Dataset
AI总结：
针对现有文本语音编辑忽视情感一致性的问题，提出EmoCorrector后校正方案，利用检索增强生成对齐情感，并构建ECD-TSE数据集，实验证明其显著提升情感表达。
摘要：
基于文本的语音编辑（TSE）仅使用文本修改语音，从而免除了重新录音的需要。然而，现有的TSE方法主要关注合成语音片段的内容准确性和声学一致性，往往忽略了由文本更改引入的情感转移或不一致性问题。为解决这一问题，我们提出了EmoCorrector，一种新颖的TSE后校正方案。EmoCorrector利用检索增强生成（RAG），通过提取编辑后文本的情感特征，检索具有匹配情感的语音样本，并合成与期望情感对齐且保持说话者身份和质量的语音。为支持TSE中情感一致性建模的训练和评估，我们率先构建了用于TSE的情感校正基准数据集（ECD-TSE）。ECD-TSE的突出特点是其包含<文本，语音>配对数据，这些数据具有多样的文本变体和广泛的情感表达。在ECD-TSE上进行的主观和客观实验及全面分析证实，EmoCorrector显著增强了预期情感的表达，同时解决了当前TSE方法中情感不一致性的局限。代码和音频示例可在https://github.com/AI-S2-Lab/EmoCorrector获取。
英文摘要：
Text-based speech editing (TSE) modifies speech using only text, eliminating re-recording. However, existing TSE methods, mainly focus on the content accuracy and acoustic consistency of synthetic speech segments, and often overlook the emotional shifts or inconsistency issues introduced by text changes. To address this issue, we propose EmoCorrector, a novel post-correction scheme for TSE. EmoCorrector leverages Retrieval-Augmented Generation (RAG) by extracting the edited text's emotional features, retrieving speech samples with matching emotions, and synthesizing speech that aligns with the desired emotion while preserving the speaker's identity and quality. To support the training and evaluation of emotional consistency modeling in TSE, we pioneer the benchmarking Emotion Correction Dataset for TSE (ECD-TSE). The prominent aspect of ECD-TSE is its inclusion of $<$text, speech$>$ paired data featuring diverse text variations and a range of emotional expressions. Subjective and objective experiments and comprehensive analysis on ECD-TSE confirm that EmoCorrector significantly enhances the expression of intended emotion while addressing emotion inconsistency limitations in current TSE methods. Code and audio examples are available at https://github.com/AI-S2-Lab/EmoCorrector.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
