优化多语言文本到语音合成中的口音与情感
Optimizing Multilingual Text-To-Speech with Accents & Emotions
AI总结：
本文提出一种集成口音保留、音译保持与多尺度情感建模的TTS架构，针对印地语和印度英语优化，显著提升口音准确率与情感识别，支持实时混合代码生成。
摘要：
最先进的文本到语音（TTS）系统在单语环境中实现了高自然度，但在合成具有正确多语言口音（尤其是印度语言）和上下文相关情感的语音方面，由于当前框架中的文化细微差别差异，仍然存在困难。本文介绍了一种新的TTS架构，该架构集成了口音保留、音译保持以及多尺度情感建模，特别针对印地语和印度英语口音进行了调优。我们的方法扩展了Parler-TTS模型，通过集成语言特定的音素对齐混合编码器-解码器架构、在母语者语料库上训练的文化敏感情感嵌入层，以及结合残差向量量化的动态口音代码切换。定量测试表明，口音准确率提高了23.7%（词错误率从15.4%降至11.8%），母语听众的情感识别准确率达到85.3%，超过了METTS和VECL-TTS基线。该系统的新颖之处在于能够实时混合代码——生成诸如“Namaste, let's talk about <Hindi phrase>”之类的语句，在保持情感一致性的同时实现不间断的口音切换。对200名用户的主观评估报告显示，文化正确性的平均意见得分（MOS）为4.2/5，远优于现有的多语言系统（p<0.01）。这项研究通过展示可扩展的口音-情感解耦，使跨语言合成更加可行，并可直接应用于南亚教育科技和无障碍软件。
英文摘要：
State-of-the-art text-to-speech (TTS) systems realize high naturalness in monolingual environments, synthesizing speech with correct multilingual accents (especially for Indic languages) and context-relevant emotions still poses difficulty owing to cultural nuance discrepancies in current frameworks. This paper introduces a new TTS architecture integrating accent along with preserving transliteration with multi-scale emotion modelling, in particularly tuned for Hindi and Indian English accent. Our approach extends the Parler-TTS model by integrating A language-specific phoneme alignment hybrid encoder-decoder architecture, and culture-sensitive emotion embedding layers trained on native speaker corpora, as well as incorporating a dynamic accent code switching with residual vector quantization. Quantitative tests demonstrate 23.7% improvement in accent accuracy (Word Error Rate reduction from 15.4% to 11.8%) and 85.3% emotion recognition accuracy from native listeners, surpassing METTS and VECL-TTS baselines. The novelty of the system is that it can mix code in real time - generating statements such as "Namaste, let's talk about <Hindi phrase>" with uninterrupted accent shifts while preserving emotional consistency. Subjective evaluation with 200 users reported a mean opinion score (MOS) of 4.2/5 for cultural correctness, much better than existing multilingual systems (p<0.01). This research makes cross-lingual synthesis more feasible by showcasing scalable accent-emotion disentanglement, with direct application in South Asian EdTech and accessibility software.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
