arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.19635cs.SDcs.AI
收藏

面向流式目标说话人提取的分块交织拼接自回归语言模型

StarTSE: Towards Streaming Target Speaker Extraction via Chunk-wise Interleaved Splicing of Autoregressive Language Model

Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);The Chinese University of Hong Kong(香港中文大学);SenseTime Research(商汤科技研究院)

Shuhai Peng, Hui Lu, Jinjiang Liu, Liyang Chen, Guiping Zhong, Jiakui Li, Huimeng Wang, Haiyun Li, Liang Cao, Shiyin Kang, Zhiyong Wu

AI总结:

本文提出一种分块交织拼接方法,用于提升流式目标说话人提取的效率与稳定性,实验表明其在低延迟下保持高性能且优于离线基线。

摘要:

尽管生成模型在目标说话人提取(TSE)中设定了新基准,但其对全局上下文的依赖限制了实时应用。直接适应流式场景常导致推理性能下降,因为训练与流式推理之间存在严重不匹配。为此,我们提出了首个针对流式TSE的自回归(AR)模型。我们的方法引入了分块交织拼接范式,以确保高效稳定的流式推理。为确保提取语音段的连贯性,我们设计了历史上下文细化机制,通过利用历史信息来缓解边界不连续性。在Libri2Mix实验中,AR生成基线在低延迟下表现下降,而我们的方法保持100%稳定性并具有优越的可懂度。此外,我们的流式结果与或优于离线基线。此外,我们的模型在消费级GPU上实现了0.248的实时因子(RTF)。本工作通过分块交织拼接范式提供了实证证据,证明AR生成主干在对延迟敏感的应用中是可行的。

英文摘要:

While generative models have set new benchmarks for Target Speaker Extraction (TSE), their inherent reliance on global context precludes deployment in real-time applications. Direct adaptation to streaming scenarios often leads to catastrophic inference performance degradation due to the severe mismatch between training and streaming inference. To bridge this gap, we present the first autoregressive (AR) models tailored for streaming TSE. Our approach introduces a Chunk-wise Interleaved Splicing Paradigm that ensures highly efficient and stable streaming inference. To ensure the coherence between the extracted speech segments, we design a historical context refinement mechanism that mitigates boundary discontinuities by leveraging historical information. Experiments on Libri2Mix show that while AR generative baseline exhibits performance degradation at low latencies, our approach maintains 100% stability and superior intelligibility. Furthermore, our streaming results are comparable to or even surpass offline baselines. Additionally, our model achieves a Real-Time-Factor (RTF) of 0.248 on consumer-level GPUs. This work provides empirical evidence that AR generative backbones are viable for latency-sensitive applications through the Chunk-wise Interleaved Splicing Paradigm.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑