arXiv 2506.220232025-06-27cs.SDcs.CLeess.AS

具有动态分块预测策略的鲁棒高效自回归语音合成

Robust and Efficient Autoregressive Speech Synthesis with Dynamic Chunk-wise Prediction Policy

X-LANCE Lab, School of Computer Science（X-LANCE实验室，计算机科学学院）；MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University（人工智能MOE重点实验室，人工智能学院，上海交通大学）；MoE Key Lab of Artificial Intelligence, Jiangsu Key Lab of Language Computing（人工智能MOE重点实验室，江苏省语言计算重点实验室）

Bohan Li, Zhihan Li, Haoran Wang, Hanglei Zhang, Yiwei Guo, Hankun Wang, Xie Chen, Kai Yu

备注：: 17 pages, 8 figures, 5 tables

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出动态分块自回归语音合成框架DCAR，通过多词元预测训练引入块到帧注意力机制并动态调整预测跨度，显著提升合成可懂度与推理速度，为下一代语音合成奠定基础。

摘要：

近期，自回归（AR）语言模型已成为语音合成的主导方法，提供了富有表现力的生成和可扩展的训练。然而，依赖下一词元预测范式的传统AR语音合成模型在处理长语音序列时经常遇到重大挑战。这些模型通常难以构建稳定的帧到帧注意力，导致延迟增加和合成质量下降，从而限制了其在实时应用中的可行性。为了解决这些局限性，我们提出了一种新颖的动态分块自回归合成框架，称为DCAR，旨在增强AR语音生成的效率和可懂度鲁棒性。DCAR通过多词元预测训练引入了块到帧注意力机制，利用在策略上训练的轻量级模块，在可变语音上下文中实现动态分块预测。DCAR动态调整词元预测跨度，显著降低了序列长度依赖性，同时获得了高合成质量。全面的实证评估表明，DCAR大幅优于传统的下一词元预测模型，在测试集上同时实现了高达72.27%的可懂度提升和2.61倍的推理加速。此外，我们进行了全面分析，以支持其作为下一代语音合成系统的通用基础。

英文摘要：

Recently, autoregressive (AR) language models have emerged as a dominant approach in speech synthesis, offering expressive generation and scalable training. However, conventional AR speech synthesis models relying on the next-token prediction paradigm often encounter significant challenges when handling long speech sequences. These models often struggle to construct stable frame-to-frame attention, leading to increased latency and degraded synthesis quality, thereby limiting their feasibility for real-time applications. To address these limitations, we introduce a novel dynamic chunk-wise autoregressive synthesis framework, termed DCAR, designed to enhance both efficiency and intelligibility robustness in AR speech generation. DCAR introduces a chunk-to-frame attention mechanism through training with multi-token prediction, enabling dynamic chunk prediction in variable speech contexts using a lightweight module trained on-policy. DCAR dynamically adjusts the token prediction span, significantly reducing the sequence length dependency while obtaining high synthesis quality. Comprehensive empirical evaluations demonstrate that DCAR substantially outperforms traditional next-token prediction models, achieving up to 72.27% intelligibility improvement and 2.61x inference speedup simultaneously on the test set. Furthermore, we conduct comprehensive analysis to support it as a versatile foundation for next-generation speech synthesis systems.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。