解混人群:从混合衍生多说话人嵌入中学习持久说话人表示
Unmixing The Crowd: Learning Persistent Speaker Representations from Mixture-Derived Multi-Speaker Embeddings
AI总结:
提出教师-学生框架,仅使用短重叠片段和置换不变潜在监督学习混合衍生多说话人嵌入,结合轻量在线记忆机制实现长时说话人重识别。
摘要:
我们研究是否可以直接从局部重叠语音混合中提取持久的对话说话人结构。我们提出一个教师-学生框架,仅使用短重叠片段和置换不变潜在监督学习混合衍生的多说话人嵌入。尽管从未明确训练用于说话人跟踪、说话人日志或对话记忆,所学的嵌入空间在推理时结合轻量在线记忆机制支持长时说话人重识别。我们进一步观察到,在未见过的重叠基数下,所学表示保留了有意义的说话人结构。我们还表明,与直接从混合中预测的嵌入相比,从分离优先流水线中提取的嵌入表现出退化的聚类结构。最后,所学嵌入在多个架构的下游目标说话人提取任务中仍然有效。这些发现表明,当与轻量推理时记忆整合相结合时,局部混合衍生表示支持持久的对话说话人重识别。
英文摘要:
We study whether persistent conversational speaker structure can be extracted directly from local overlapping speech mixtures. We propose a teacher-student framework that learns mixture-derived multi-speaker embeddings using only short overlapping segments and permutation-invariant latent supervision. Despite never being explicitly trained for speaker tracking, diarization, or conversational memory, the learned embedding space supports long-form speaker re-identification when combined with a lightweight online memory mechanism during inference. We additionally observe that the learned representation retains meaningful speaker structure under unseen overlap cardinalities. We further show that embeddings extracted from separation-first pipelines exhibit degraded clustering structure compared to embeddings predicted directly from mixtures. Finally, the learned embeddings remain effective for the downstream target speaker extraction task across multiple architectures. These findings suggest that local mixture-derived representations support persistent conversational speaker re-identification when combined with lightweight inference-time memory consolidation.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。