Lina-Speech：用于多样本提示文本转语音合成的门控线性注意力与初始状态调优
Lina-Speech: Gated Linear Attention and Initial-State Tuning for Multi-Sample Prompting Text-To-Speech Synthesis
AI总结：
针对神经编解码器语言模型在TTS语音克隆中存在的上下文长度有限、韵律情绪适配难及推理吞吐量低等问题，提出Lina-Speech模型，采用GLA替代自注意力并结合IST策略，实现多样本条件控制，提升性能与吞吐量，支持细粒度特征控制。
摘要：
基于Transformer架构构建的神经编解码器语言模型彻底革新了文本转语音（TTS）合成，通过将语音克隆视为前缀延续任务而在该领域表现出色。然而，其有限的上下文长度限制了对短语音样本的处理效果，导致语音克隆能力在说话人韵律和风格的覆盖范围与多样性上存在局限。此外，从短前缀中适配韵律、口音或恰当情绪仍是一项具有挑战性的任务。最后，自注意力的二次复杂度限制了推理吞吐量。在本研究中，我们提出Lina-Speech模型，它采用门控线性注意力（GLA）替代标准自注意力作为核心架构，在匹配最先进性能的同时提升了推理吞吐量。利用循环架构的状态性特征，我们引入初始状态调优（IST）策略，该策略实现了对任意数量和长度的多语音样本条件控制的可能性，并为语音克隆以及域外说话风格与情绪适配提供了全面且高效的解决方案。我们通过实验证明了该方法在控制韵律、情绪等细粒度特征方面的有效性。代码、检查点和演示可免费获取：https://github.com/theodorblackbird/lina-speech
英文摘要：
Neural codec language models, built on transformer architecture, have revolutionized text-to-speech (TTS) synthesis, excelling in voice cloning by treating it as a prefix continuation task. However, their limited context length hinders their effectiveness to short speech samples. As a result, the voice cloning ability is restricted to a limited coverage and diversity of the speaker's prosody and style. Besides, adapting prosody, accent, or appropriate emotion from a short prefix remains a challenging task. Finally, the quadratic complexity of self-attention limits inference throughput. In this work, we introduce Lina-Speech, a TTS model with Gated Linear Attention (GLA) to replace standard self-attention as a principled backbone, improving inference throughput while matching state-of-the-art performance. Leveraging the stateful property of recurrent architecture, we introduce an Initial-State Tuning (IST) strategy that unlocks the possibility of multiple speech sample conditioning of arbitrary numbers and lengths and provides a comprehensive and efficient strategy for voice cloning and out-of-domain speaking style and emotion adaptation. We demonstrate the effectiveness of this approach for controlling fine-grained characteristics such as prosody and emotion. Code, checkpoints, and demo are freely available: https://github.com/theodorblackbird/lina-speech
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
