基于短上下文说话人嵌入的低延迟多说话人跟踪研究
Towards Low-Latency Tracking of Multiple Speakers With Short-Context Speaker Embeddings
AI总结：
针对常见说话人嵌入提取器处理短时上下文和重叠语音的不足，提出基于知识蒸馏的短上下文嵌入提取方法，结合逐块身份重新分配，提升低延迟多说话人跟踪性能，蒸馏模型在短上下文和重叠鲁棒性上表现更优。
摘要：
说话人嵌入是一种很有前景的身份相关特征，可通过利用跟踪系统的空间预测（即执行身份重新分配）来提升其身份分配性能。常见的说话人嵌入提取器通常难以处理短时上下文和重叠语音，这导致需要长期身份重新分配以利用更长的时间上下文。然而，这会增加跟踪系统错误的概率，进而对身份重新分配产生负面影响。为此，我们提出一种基于知识蒸馏（KD）的训练方法，用于从两个说话人混合语音中提取短上下文说话人嵌入。我们利用波束成形技术获取目标说话人的空间信息以减少重叠。我们研究了对固定大小块执行身份重新分配（即逐块身份重新分配）的可行性，以构建低延迟的基于说话人嵌入的跟踪系统。结果表明，我们的蒸馏模型在短上下文嵌入提取方面有效，且对重叠更鲁棒。不过，逐块重新分配的结果显示，还需要进一步工作来更有效地处理同时说话的情况。
英文摘要：
Speaker embeddings are promising identity-related features that can enhance the identity assignment performance of a tracking system by leveraging its spatial predictions, i.e, by performing identity reassignment. Common speaker embedding extractors usually struggle with short temporal contexts and overlapping speech, which imposes long-term identity reassignment to exploit longer temporal contexts. However, this increases the probability of tracking system errors, which in turn impacts negatively on identity reassignment. To address this, we propose a Knowledge Distillation (KD) based training approach for short context speaker embedding extraction from two speaker mixtures. We leverage the spatial information of the speaker of interest using beamforming to reduce overlap. We study the feasibility of performing identity reassignment over blocks of fixed size, i.e., blockwise identity reassignment, to go towards a low-latency speaker embedding based tracking system. Results demonstrate that our distilled models are effective at short-context embedding extraction and more robust to overlap. Although, blockwise reassignment results indicate that further work is needed to handle simultaneous speech more effectively.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
