基于记忆感知多说话人嵌入与序列到序列架构的神经说话人日志
Neural Speaker Diarization Using Memory-Aware Multi-Speaker Embedding with Sequence-to-Sequence Architecture
AI总结：
提出NSD-MS2S神经说话人日志系统，融合记忆感知多说话人嵌入与序列到序列架构，在CHiME-7 EVAL集上DER达15.9%，相对提升49%，并引入深度交互模块增强嵌入判别性。
摘要：
我们提出了一种新颖的神经说话人日志系统，该系统采用记忆感知多说话人嵌入与序列到序列架构（NSD-MS2S），整合了记忆感知多说话人嵌入（MA-MSE）和序列到序列（Seq2Seq）架构的优势，从而在效率和性能上均有所提升。接下来，我们通过引入输入特征融合进一步降低解码过程中的内存占用，并采用多头注意力机制来捕获不同层次的特征。NSD-MS2S在CHiME-7 EVAL集上实现了15.9%的宏观日志错误率（DER），相比官方基线系统相对提升了49%，这也是我们在CHiME-7 DASR挑战赛主赛道中取得最佳性能的关键技术。此外，我们在MA-MSE模块中引入了一个深度交互模块（DIM），以更好地提取更干净、更具判别性的多说话人嵌入，使当前模型优于我们在CHiME-7 DASR挑战赛中所使用的系统。我们的代码将在https://github.com/liyunlongaaa/NSD-MS2S上提供。
英文摘要：
We propose a novel neural speaker diarization system using memory-aware multi-speaker embedding with sequence-to-sequence architecture (NSD-MS2S), which integrates the strengths of memory-aware multi-speaker embedding (MA-MSE) and sequence-to-sequence (Seq2Seq) architecture, leading to improvement in both efficiency and performance. Next, we further decrease the memory occupation of decoding by incorporating input features fusion and then employ a multi-head attention mechanism to capture features at different levels. NSD-MS2S achieved a macro diarization error rate (DER) of 15.9% on the CHiME-7 EVAL set, which signifies a relative improvement of 49% over the official baseline system, and is the key technique for us to achieve the best performance for the main track of CHiME-7 DASR Challenge. Additionally, we introduce a deep interactive module (DIM) in MA-MSE module to better retrieve a cleaner and more discriminative multi-speaker embedding, enabling the current model to outperform the system we used in the CHiME-7 DASR Challenge. Our code will be available at https://github.com/liyunlongaaa/NSD-MS2S.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
