arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.05787eess.AScs.CL
收藏

MARS6:一种小型且鲁棒的分层编解码器文本转语音模型

MARS6: A Small and Robust Hierarchical-Codec Text-to-Speech Model

Stellenbosch University(斯坦陵布什大学);Camb.ai Inc.(Camb.ai公司)

Matthew Baas, Pieter Scholtz, Arnav Mehta, Elliott Dyson, Akshat Prakash, Herman Kamper

AI总结:

本文提出小型鲁棒的分层编解码器 TTS 模型 MARS6,利用 12 Hz 分层解码器及多种训练推理技术,以 7000 万参数达到媲美数倍大模型的语音合成与零样本克隆效果。

摘要:

基于编解码器的文本转语音(TTS)模型凭借零样本声音克隆能力展现出令人瞩目的音质。然而,它们在处理更具表现力的参考音频或复杂文本输入时往往表现不佳。我们提出 MARS6,这是一款用于快速、富有表现力的 TTS 的鲁棒编码器-解码器 Transformer。MARS6 建立在语音语言建模的最新进展之上。通过为其解码器采用分层结构,新语音 token 仅以 12 Hz 的速率处理,从而能够在保持重建质量的同时高效建模长文本。我们结合了若干近期的训练与推理技术,以减少重复生成并提升输出的稳定性与质量。这使得拥有 7000 万参数的 MARS6 能够达到与体积大数倍的模型相近的性能。我们在客观与主观评估中,通过比较 TTS 输出质量和参考说话人克隆能力证明了这一点。项目页面:https://camb-ai.github.io/mars6-turbo/

英文摘要:

Codec-based text-to-speech (TTS) models have shown impressive quality with zero-shot voice cloning abilities. However, they often struggle with more expressive references or complex text inputs. We present MARS6, a robust encoder-decoder transformer for rapid, expressive TTS. MARS6 is built on recent improvements in spoken language modelling. Utilizing a hierarchical setup for its decoder, new speech tokens are processed at a rate of only 12 Hz, enabling efficient modelling of long-form text while retaining reconstruction quality. We combine several recent training and inference techniques to reduce repetitive generation and improve output stability and quality. This enables the 70M-parameter MARS6 to achieve similar performance to models many times larger. We show this in objective and subjective evaluations, comparing TTS output quality and reference speaker cloning ability. Project page: https://camb-ai.github.io/mars6-turbo/

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑