arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2609.12728eess.AS

X-Pred MeanFlow:用于流式令牌到梅尔频谱语音解码的方法

X-Pred MeanFlow for Streaming Token-to-Mel Speech Decoding

Hanke Xie, Xiaming Ren, Qirui Zhan, Jingbin Hu, Wenhao Li, Haoyu Zhang, Ruonan You, Chengyou Wang, Yunxiang Chen, Houdun Liu, Su Feng, Lei Xie

AI总结:

提出X-Pred MeanFlow,通过梅尔空间预测重新参数化MeanFlow,实现少步流式令牌到梅尔频谱解码,提升合成质量并支持稳定流式生成。

AI中文摘要:

近期在基于离散令牌的语音生成方面的进展,凸显了在流式和对话场景中高效令牌到波形合成的重要性。流匹配声学解码器能够实现高质量的令牌到梅尔频谱生成,但其迭代采样需要多次神经函数评估,限制了低延迟语音合成。MeanFlow通过建模时间区间内的平均速度来减少采样预算,然而在极少数步骤的令牌到梅尔频谱生成中保持高声学质量仍然具有挑战性。为应对这一挑战,我们提出了X-Pred MeanFlow,一种少步流式令牌到梅尔频谱解码器,通过梅尔空间预测对MeanFlow进行重新参数化。该解码器预测一个广义梅尔场,并解析地推导出相应的平均速度用于采样,从而在保持MeanFlow公式的同时提供直接的声学预测目标。我们进一步引入了层选择性分块注意力,以实现具有有界上下文的连续分块生成。实验表明,X-Pred MeanFlow在少步令牌到梅尔频谱合成上优于Direct-$u$ MeanFlow,并支持稳定的流式生成。语音样本可在该URL获取:https://renxiaming.this http URL

英文摘要:

Recent advancements in discrete token-based speech generation have highlighted the importance of efficient token-to-waveform synthesis in streaming and dialogue scenarios. Flow-matching acoustic decoders achieve high-quality token-to-mel generation, but their iterative sampling requires multiple neural function evaluations, limiting low-latency speech synthesis. MeanFlow reduces the sampling budget by modeling the average velocity over a temporal interval, yet maintaining high acoustic quality under extremely few-step token-to-mel generation remains challenging. To address this challenge, we propose X-Pred MeanFlow, a few-step streaming token-to-mel decoder that reparameterizes MeanFlow with mel-space prediction. The decoder predicts a generalized mel field and analytically derives the corresponding average velocity for sampling, thereby preserving the MeanFlow formulation while providing a direct acoustic prediction target. We further introduce layer-selective block-wise attention to enable continuous chunk-wise generation with bounded context. Experiments show that X-Pred MeanFlow improves few-step token-to-mel synthesis over Direct-$u$ MeanFlow and supports stable streaming generation. Speech samples are available.https://renxiaming.github.io/xpred-meanflow-stream-demo

补充信息

↑