arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.01316eess.AS
收藏

跨说话人相对线索用于两阶段文本引导的目标语音提取

Inter-Speaker Relative Cues for Two-Stage Text-Guided Target Speech Extraction

Tampere University(坦佩雷大学)

Wang Dai, Archontis Politis, Tuomas Virtanen

AI总结:

本文提出一种两阶段文本引导的目标语音提取框架,利用相对线索提升分类和提取性能,实验表明相对线索在分类准确性和提取效果上优于独立线索。

摘要:

本文研究了相对线索在基于文本的目标语音提取(TSE)中的应用。我们首先从人类感知和标签量化的角度提供相对线索的理论依据,证明相对线索保留了连续值属性中通常在绝对分类表示中丢失的细粒度区别。在此分析基础上,我们提出一个两阶段TSE框架,其中语音分离模型首先生成候选源,随后文本引导的分类器根据嵌入相似性选择目标说话人。在该框架内,我们训练了两个独立的分类模型,以评估相对线索在连续值属性情况下优于独立线索的优势,考虑了分类准确性和TSE性能。实验结果表明:(i)相对线索在总体分类准确性和改进的TSE性能上优于独立线索;(ii)所提出的两阶段框架在信号级和客观感知度量上显著优于单阶段文本条件提取方法;(iii)几种相对线索,包括语言、响度、距离、时间顺序、说话持续时间、随机线索和所有线索,甚至可以超越基于注册音频的TSE系统。进一步分析揭示了不同线索类型的判别能力存在显著差异,为不同相对线索在TSE中的有效性提供了见解。

英文摘要:

This paper investigates the use of relative cues for text-based target speech extraction (TSE). We first provide a theoretical justification for relative cues from the perspectives of human perception and label quantization, showing that relative cues preserve fine-grained distinctions that are often lost in absolute categorical representations for continuous-valued attributes. Building on this analysis, we propose a two-stage TSE framework in which a speech separation model first generates candidate sources, followed by a text-guided classifier that selects the target speaker based on embedding similarity. Within this framework, we train two separate classification models to evaluate the advantages of relative cues over independent cues in case of continuous-valued attributes, considering both classification accuracy and TSE performance. Experimental results show that (i) relative cues achieve higher overall classification accuracy and improved TSE performance compared with independent cues; (ii) the proposed two-stage framework substantially outperforms single-stage text-conditioned extraction methods on both signal-level and objective perceptual metrics; and (iii) in controlled target-selection comparisons, the all cues, random cues, and temporal order settings outperform the strong enrollment-audio selection based on ECAPA-TDNN speaker embeddings, while language and distance cues remain competitive. In the WHAM!-noise evaluation, textual cues also show strong noise robustness, while all cues, random cues, and gender are especially robust. These results demonstrate the practical value of textual cues when enrollment speech is unavailable or cannot be used due to privacy concerns. Further analysis reveals notable differences in discriminative power across cue types, providing insights into the effectiveness of different relative cues for TSE.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑