基于任务向量算术的语言模型文本到语音情感表达控制
Task-Vector Arithmetic for Emotional Expressivity Control in Language-Model-Based Text-to-Speech
AI总结：
本文通过系统消融实验定位情感韵律的主要载体为x-vector，并提出一种基于x-vector质心算术的无训练方法，实现跨说话人情感强度控制，在保留身份和可懂度的同时提升情感相似度。
摘要：
我们研究了任务向量算术（在模块化文本到语音（TTS）中成功用于跨说话人情感强度控制）是否能够迁移到基于语言模型骨干和上下文学习（LM-TTS）构建的大规模TTS系统。通过在Qwen3-TTS-12Hz-1.7B上对四个逐渐缩小的操作数——通过LoRA微调的模型权重、连续编解码器嵌入、离散编解码器标记以及由ECAPA-TDNN编码器（与合成骨干联合训练）生成的说话人嵌入（x-vector）——进行系统消融研究，我们将情感韵律的主要载体定位到x-vector。基于这一发现，我们提出了一种基于x-vector空间质心算术的无训练方法：情感方向τ = E_i[x(s_i, emo)] - E_i[x(s_i, neutral)]，应用于未见过的目标说话人：x_new = x(target, neutral) + α·τ。使用ESD（英语）作为τ源，emoUERJ（巴西葡萄牙语）作为跨语言真实目标，我们观察到在英语保留说话人上，情感余弦相似度比ICL基线平均提升+0.29，在巴西葡萄牙语保留说话人上提升+0.09，同时很大程度上保留了身份（多说话人τ变体的WavLM SECS ≥ 0.88）和可懂度（PT-BR中WER ≈ 0）。这些结果初步证明，当算术操作作用于说话人嵌入时，可以规避先前报道的基于质心算术的风格控制与基于标记的TTS架构不兼容的问题。
英文摘要：
We investigate whether task-vector arithmetic, successful for cross-speaker emotional intensity control in modular text-to-speech (TTS), transfers to large-scale TTS systems built on language-model backbones with in-context learning (LM-TTS). Through a systematic elimination study over four progressively narrower operands on Qwen3-TTS-12Hz-1.7B - model weights via LoRA fine-tuning, continuous codec embeddings, discrete codec tokens, and the speaker embedding (x-vector) produced by an ECAPA-TDNN encoder jointly trained with the synthesis backbone - we localize the dominant carrier of emotional prosody to the x-vector. Building on this finding, we propose a training-free method based on centroid arithmetic in x-vector space: an emotion direction $τ= \mathbb{E}_i[x(s_i,\text{emo})] -\mathbb{E}_i[x(s_i,\text{neutral})]$ applied to an unseen target speaker as $x_{\text{new}} = x(\text{target},\text{neutral}) + α\cdotτ$. Using ESD (English) as the $τ$ source and emoUERJ (Brazilian Portuguese) as a cross-lingual ground-truth target, we observe average gains of $+0.29$ in emotion2vec cosine over the ICL baseline on English held-out speakers and $+0.09$ on Brazilian Portuguese held-out speakers, while largely preserving identity (WavLM SECS $\gtrsim 0.88$ for the multi-speaker $τ$ variant) and intelligibility (WER $\approx 0$ in PT-BR). These results offer initial evidence that the dominant carrier of emotional prosody in this class of models is localizable, by elimination, to the co-trained speaker embedding, where training-free centroid arithmetic remains effective even under cross-lingual transfer.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
